access_ScottWalesEmail20April2012 - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki
Matthews, David [[email protected]]
[Reply] [Reply All] [Forward]
Actions
To:
Scott Wales
Cc:
Michael Rezny [[email protected]]; Michael Naughton [[email protected]]; Asri Sulaiman [[email protected]]; Shin, Matthew [[email protected]]
Friday, 20 April 2012 2:21 AM
Scott
Many thanks for the feedback.
> I'm part of the UM support group for the Australian
> universities and am
> currently maintaining our test installation of Rose. The
> suite database
> proposal on the collab wiki has recently been brought to my
> attention, I wanted
> to share some thoughts on it with you.
>
> The current proposal is to continue to use a fixed width alphanumeric
> identifier for jobs. Is there a particular reasoning for
> choosing this as
> opposed to simply using a number incremented for each job, eg
> 1,2,3?
Using a fixed width just makes it easier for sorting and
organising id's.
> A drawback
> of the current fixed width system is the limited amount of
> runs that can be
> grouped into a folder which we've hit when creating nested runs.
In the new system you will use the discovery info to group suites
together so there are no such restrictions.
Also, some of the cases where you currently have multiple jobs
are likely to be a single suite using Rose.
> I personally believe it would be better to have the job names
> in the repository
> more free-form, with users able to group jobs as they wish
> under a username
> folder. While id numbers can be convenient for quick
> reference this could be
> handled by the metadata database. You also avoid the fcm
> magic involved in
> creating a new folder with the correct id number in the repository.
>
> Having branches within jobs seems to me a bit excessive.
> Perhaps it would be
> better to think of the jobs themselves as branches from a set
> of standard runs
> for NWP, climate &c., with the ability to merge jobs provided.
There's a section in the notes which tries to give examples of
where we think branches will be useful.
> Storing jobs in a sub directory for each character of the ID
> also might be a
> bit much. Perhaps this could be condensed into something like
> ab/012 rather
> than a/b/0/1/2.
We did consider this but decided splitting each character is
simpler. It doesn't really matter if you accept that you will
normally locate suites using the discovery database rather
than browsing the repository.
> Have you considered using svn properties to store the
> metadata rather than a
> config file? I believe there is an api to access properties
> that a discovery
> tool could be built around. This would avoid having to
> synchronise to an
> external database, although it would be less convenient to
> modify than a text
> file.
The database is essential in order to provide a fast
discovery interface. Updating the database is simple via a
post-commit hook script.
> I think it would be worthwhile to limit the amount of magic
> done by fcm,
> preferably it would just be a thin layer on top of subversion
> adding support
> for path shortening. If the metadata stuff is done using
> commit hooks that
> shouldn't be a problem.
>
> Suite-create could be handled as creating a branch off of a
> standard empty job,
> avoiding needing to add logic for creating the directory
> structure to fcm.
This would mean that you can't create branches of suites which
we think will be important.
> What is the purpose of having a standard location for
> checking out a job,
> rather than just adding it to the current directory? Users
> should be encouraged
> to think of working copies as temporary things, the permanent
> storage is the
> repository itself.
This will be optional. However, we think supporting a common
best practise will make life a lot easier and allow use to
provide tools which take advantage of this.
> In order to properly share jobs between collaborators you
> need to share not
> just the UM configuration, but also the ancillary and IC
> files. The best way to
> do that might require some thought.
Agreed. This is being thought about separately (it's not
really a Rose issue).
> I think I'll leave it there for the moment as this has gotten rather
> long-winded, hopefully at least some of this is useful to
> you. I'm very interested
> in the direction that Rose is heading, and definitely think
> that it's a positive one.
Thanks - we're certainly excited by the possibilities.
I've just put a new version of our proposal on the wiki which
addresses some of your concerns so please take a look at it
in addition to my brief responses above.
Thanks,
Dave