Meeting Notes for 26 September 2007 Hardware SC07 travel plans, brochures NFS at boot (and others) Storage issues Caching metadata (Mihael) Refactored Code (Mihael) Statistics on use (Bob) Forcing presurvey

Meeting Notes for 26 September 2007

Present: Eric, Mihael, TomL, Bob, Mike
Absent: TomJ, Tibi

Hardware

Server is on the way to Argonne (Bob); Mike ordered more storage as well.

SC07 travel plans, brochures

We will ask for 500 i2u2 brochures if we are not paying for it; The URL for brochure is: http://ed.fnal.gov/work/SC2007/i2u2_brochure.zip [220MB!]

Mihael and Eric have been collaborating off-line and making good progress on the LIGO analysis, so Eric will use his one allocated trip to go to SC2007 instead of Chicago.

NFS at boot (and others)

NFS does not start at boot. All machines come up at run level 2; should come up at run level 3. Mike W. suggested that systems should take ownership of doing this. We need to have NFS at mount. Eric reported installing a new apache server on www13, and he would want systems to take ownership of that. It was noted that the sequence of starting up everything is important. Eric will make sure Systems gets it right.

Eric: We should schedule a time to test the steps to make sure everything is included in the procedures. This would include doing a reboot of machine(s) to verify that it works, which would have to be scheduled.

Our Rollout may be out of synch. Need to make sure all the load-balancing machines the same. Need to boot data0 and data1 first. Back when they lost their DHCP they did not mount the remote filesystems. Mihael thinks they will be o.k if they are brought up in the standard way. Need a checklist to check things. Should make a wiki page of what needs doing. Not in ClusterOps; make its own page, but link to it from ClusterOps

Action Item: Bob will make first pass on checklist. He can invite other people to improve it.
Action item: Eric will contact Systems about getting NFS to start at boot.

Storage issues

The new machine will solve some of this. It is going to be used for all the data. It has 3 Terabytes. Cosmic data is currently triply backed up. Eric is looking for storage space for LIGO data. LIGO data won't need triple redundancy, because if there is a disk failure it can be recovered from Hanford. Currently he has 1G on a temporary place for LIGO. Minute trends could fit on a 75GB partition on data1 called /data, which appears to be empty and is not exported to the cluster. For minute trends he needs 15GB for existing data, so 30 GB to allow for growth, and 60GB if he adds data from Livingston.

Before the meeting Eric sent around a summary of what he'd found about disk storage and RAID arrays. That is attached as

• DiskStorageIssues: Notes on unused disk space and RAID arrays

In one week Eric will use the 75 GB unless someone objects. He will also have to get data for second trends eventually. He will give us an estimate of what he needs for second trends.

How much will Star and Amelia need? We decided we want to know, but don't need to know right now.

Caching metadata (Mihael)

A by-product of the new API. Mihael discovered that the largest part of the time it takes to search data is in getting the metadata. He has set up a metadata cache for bulk searching. Specific queries will not return the cache metadata. Deployed on www12. This method cannot be applied to production because it relies on the .... ?. This is just for the data files, not plots. As Mihael said: The achievement was getting all (well, almost all) of the data catalog stuff going through an abstract layer in which the caching can be implemented efficiently.

Refactored Code (Mihael)

Status Report: have to copy the production database over to www12. Tibi will need to do this.

Statistics on use (Bob)

Marge would like to see more statistics that relate to current use. Bob can list the number of research groups, but not all of these are active; they may be left over from former years. We can search for research groups with AY2007, but that may leave out groups that are reused by teachers from year to year.

Action item: Bob will make a list of what the data he wants to collect.

Tibi also has a tool that he is using; tool works for www11; Bob runs an app. awstats followed by the URL. It would be nice to concatenate the results for all machines.

Action Item: Bob will talk to Marge about the pre-survey issue.

Forcing presurvey

Marge thinks we need to force teachers to have their students take the pre-survey so that we can get enough data for our evaluator. Eric suggested only forcing 50% to take it (chosen at random), or some kind of prize for taking it ( redeemable coupon? ) Liz: If you are going to do research on the effectiveness of our project, we do need this data. Tom L and Mihael thought it was onerous to require this, especially if it prevented the group from using the e-Lab until everyone had taken the test. What if someone was sick? The teacher could delete that person from the group. Eric brought up the issue that we should be tracking individual students rather than groups. The e-Labs were really designed for collaborative groups, not individuals so individual students are only tracked with the surveys. This discussion is probably more appropriate for the Education developers.

After the meeting Marge suggested that this might be a good topic for the Fellows to discuss.

-- Main.LizQuigg - 26 Sep 2007
-- Main.EricMyers - 28 Sep 2007