In this post I’ll report on running the application “mc09_7TeV.107691….” from the GridPilot app store. In the case of NorduGrid and WLCG, the ATLAS software is preinstalled on the resources. In the case of GridFactory, the jobs run inside a CernVM appliance with ATLAS software loaded through the AFS network file system. The input dataset consisted of 26 files totaling … Read the rest
In previous posts we saw that I/O bound jobs ran ~3 faster on standard SATA disks than on network file systems and block devices (GPFS, NFS, EBS). This post reports on CPU bound jobs. I ran standard ATLAS Monte Carlo simulation on both grid and cloud resources: imported the ATLAS simulation app and ran the default 100 small jobs, each … Read the rest
In this post, I’ll take a look at some more runs of the “atlas_d3pd_boildown” application available in the GridPilot app store. The difference w.r.t. the runs described in a previous post is that this time I ran on cloud as opposed to grid resources. On dedicated hardware and on two public clouds, Amazon’s EC2 and Cabo’s Irigo cloud, I … Read the rest
Professionally, I recently had to set up some improvised storage, making use of 6 machines, each with a 1 TB disk that was not used for anything else. Preferably with a common name space. Pooling such 6 disks into a common storage solution may sound like a common, mundane task, but there does not appear to be any open-source … Read the rest
In this post, I’ll take a look at some runs of the POV-Ray application available in the GridPilot app store: To import this app, just choose “File → Import application”, navigate to the relevant folder and click “OK”.
This example is special in that it does not depend on any preinstalled software package (runtime environment), but includes a precompiled binary. This binary will of course only for certain run on the system it was compiled on. We compiled on Debian Sarge and Scientific Linux 5 and run on all back-ends: a local virtual machine, GridFactory without virtualization and … Read the rest
Here is a video I put together to demo how to use GridPilot to run computations on a GridFactory cluster:
The demo uses the default input files – which are 12 royalty free music files found on incompetech.com. This can be changed – by right-clicking on the input dataset, “music_files”, and choosing “Import file(s)”. If you’ve already imported the … Read the rest
Given the popularity of the iPhone, an interesting use of a batch system is conversion of movie files from the AVI to the MPG4 format. In this post I’ll explain 3 ways doing this with GridPilot. Which way you prefer will likely depend on the number and size of files you want to convert and the power … Read the rest
To gauge the performance of both GridFactory and virtualization layers in a high-CPU/low-throughput setting, we chose the standard ray-tracing program POV-Ray and a standard benchmarking image, shipped with the program.
The standard image that was rendered.
This example is a fairly naive benchmarking exercise consisting simply in rendering the same image with POV-Ray 20 times. Each POV-Ray job used a … Read the rest
This example demonstrates the use of GridPilot in data processing in high energy physics (HEP). It makes extensive use of some HEP-specific technologies, that are incapsulated in GridPilot in the form of plugins: the ATLAS DB plugin and the NG and GLite computing system plugins. The jobs chosen are so-called … Read the rest
Canada was great – here’s a couple of pictures from the Summit photo contest:
My talk was about what I see as an opportunity to dramatically simplify and improve the infrastructure of scientific computation and collaboration.
I … Read the rest
In the context of the Nordic HPC community, I’ve been involved in some discussions on the applicability of cloud computing in HPC. Also in the blogosphere, the subject is receiving some attention (e.g. at bigdatamatters.com and hpcwire.com).
Here are some comments of mine:
A few times recently I’ve come across an argument that can be summarized by the following … Read the rest
Last week I gave a general talk on cloud computing at the yearly conference “Softwareudvikling på Tværs”, arranged by Teknologisk Institut. The purpose was to inform about what (I consider) cloud computing is, where it is currently used and where it is headed.
The arrangement was well organized and included many interesting talks.
This is a follow-up to a previous post.
VENUE: Niels Bohr Institute, Blegdamsvej 17, 2100, Copenhagen
TIME: May 12, 2009, 14:00
14:10 – 14:40, “Cloud and Virtualization to Support Grid Computing”,
Ass. Professor Ruben S. Montero , DSA research group, Universidad de Complutense, Madrid
14:40 – 15:00, … Read the rest
Congratulations to the organizers and Fukuoka Institute of Technology for a pleasant and well-organized ARES 2009 conference.
Almost all the sessions I attended had engaging discussions.
If you’re interested in grid security you might want to read a piece I and 4 colleagues wrote on our gripes with the current model (GSI, proxy certificates and all … Read the rest
To inform some of the stakeholders about the background and scope of the project “grid.dk”, I wrote a piece featured on International Science Grid This Week.
The message is that we are aware of the the difficulties other projects have had in attracting users. Our take at this will be to experiment with new user interfaces and focus on … Read the rest
Just came across this page. So the Open Group actually has a “Batch Environment Services and Utilities” specification. As far as i can see GridFactory implements a good set of the specified services and utilities.
UPDATE: mod_gacl is now distributed with GridFactory
Ever wanted to:
- use Apache as a secure file server,
- use X.509 certificates for authenticating both server and client,
- control who can put and get files in each directory?
Well, then go right ahead and download mod_gacl.
Installation: Follow the instructions in the README file. Since an … Read the rest