Using The Globus Commands

Starting on the Project home page, seen (annotated) below, you can use the Globus commands to upload and download files; and obtain the status of the upload process. To get to the project home page from the Projects list page, click on the project name, which will take you to the Project home page. If you already on the Project page, click on the "home" button, the project name at the top of the left side navigation panel.

On the Project Home page you will see the globus commands (1) and the your user name (2):

We will discuss the Globus Commands (Upload, Download, and Status ), below, but first we need to talk about setting your Globus name in your User Profile.

Set up Globus Name in User Profile

To get to you user profile, click on your user name, in the upper right hand corner of the Project page, shown as (2), above. When you do, you will see this pop-up menu, select ACCOUNT SETTINGS:

In the account settings panel, scroll to the Globus User Name entry, enter you Globus User Name (usually you email address) and click "UPDATE GLOBUS USER NAME"

For further details on setting up the Globus interface, and your Globus user name, see the Globus help pages on installing Globus Connect Personal, and setting up your account. You can set up Globus Connect Personal on any number of machines using the same account. Or different accounts (each with it's own separate email address). If you are using different accounts, don't forget to update the Globus (ID) name in your user profile when you switch accounts.

Globus Upload Command

In the project page there are the Globus commands, shown as (1) in the image at the top of this page. The first of these is the Globus Upload Command.

You will use this upload command to transfer experiment data (or other collections of directories and files) into your project from any computer on which you have set up Globus Connect Personal. Don't forget to set your Globus username (ID) in your user profile.

To start a Globus transfer into your project in Materials Commons, click on the "Globus Upload Command"; this is will open the Globus Upload panel:

The button in the panel opens the Globus Transfer interface, or (if you are not logged in already) the Globus login panel. Logging in is straight forward, follow the steps in the interface to log in, if you are having trouble, review the Globus Help on getting started. Also, if you neglected to set up your Globus name in your user settings you will see an error message. In that case, set the information in your user profile, and try again.

Normally, however, you will see this Globus Transfer Interface:

Some things to note:

• This link is a ONE TIME USE ONLY link. It can not be reused because as soon as the initial transfer is complete the link is removed. If you need additional transfers into your project, you can simply make another Globus upload request. Request can be overlapped, but transferring the same file twice will have indeterminate results. That is, of the two versions of the file being transferred, only one will end up in the project, and there is no way to determine which one will 'win'.
• The transfer panel already has the destination filled in. The is a special folder, on the Materials Commons server, created specifically for this transfer. Don't try to change the destination folder.
• You will be setting source folder, shown on the left side of the image. If this portion of the interface is not clear, make sure that your are in the split-panel mode (the 'book', two page, icon in the 'Panels' selector ). The display should look very much like the images above, and below.

In the Globus Transfer panel, navigate to the directory that contains the files and directories that you want to select from for transfer. Select the files and/or directories that you want to transfer, one, or many. In the example shown here, we have picked the folders named, allInOne, data, and filename_test for transfer.

Having selected the files and/or folders for transfer, click the "Start" button on the bottom of the source panel, the left panel. Note, for most cases the default Transfer & Sync Options are best, you can read more about those settings in the Globus transfer documentation.

Once your transfer is request is accepted, you will see an "in panel" message like this:

Click on the link for the Task Id, to see the Globus Transfer activity panel, were you can monitor the status of the Globus transfer. You can also check the status of the Upload process using the "Status of Globus Upload" button in the left-side navigation bar of the Project page. Show here is the status response while globus is upload files to Materials Commons:

Once the globus transfer is done, Materials Commons will remove the upload link, and begin processing the files, for a small dataset the actions needed to process the files will be fairly quick. We will examine the details on the Status Command and its status responses, below. When the upload is finished the status will look something like this:

When uploading and processing are completed you can view the transferred files and/or directories in your project. Click on the AllFile button in the left-side navigation bar of the Project page, and you files will be shown. If you believe that your files should be shown and they are not, try the Sync button left-side navigation bar, which synchronizes your project data (and display) with the data on the Materials Commons server, bringing any changes (files and directories, in this case) up to date. You can also just refresh the browser page.

In this example transfer, the three transferred directories now show up in the project file directory.

Globus Status Command

Many transfers, especially those involving large amounts of data, will take a longer time to complete. In those cases, when you wish to monitor the actions being taken, the "Status of Globus Upload" button in the left-side navigation bar of the Project page will give you a rough idea of what is happening. There are two major stages in the upload process, globus transfer, and Materials Commons processing of the directories and files to make them part of your project.

To facilitate tracking these two stages of an upload Materials Commons create an upload status record for each upload request. We have already seen, in the description of the Globus Upload Command, above, now the Upload status report can be used to monitor upload. Now we explain how those status reports might differ for larger upload. When we start a longer upload, the Globus UI will report that upload in progress:

In this example, we have started a longer, second upload, so we see an older record for the first upload ("All files loaded", and the record for this upload showing (also) that the globus upload in is progress. Obviously the Globus UI shows greater detail so we suggest that your return to it, in order to see exactly how far along it has proceeded. You can always get back to the Globus activity details by going to your globus connect personal app (or web interface) and clicking on the "Activity" command or link. Then navigate to the active task or tasks and you will see a display like the one above.

Obviously the Globus UI shows greater detail so we suggest that your return to it, in order to see exactly how far along it has proceeded. You can always get back to the Globus activity details by going to your globus connect personal app (or web interface) and clicking on the "Activity" command or link. Then navigate to the active task or tasks and you will see a display like the one above.

When the globus upload phase is done, and the materials commons processing is started, you will see a status response panel like this:

Indicating that the files are being loaded into your project. Come back later and check again. Or you can click the refresh button "REPEAT STSTUS INFO REQUEST" to see the status update. When the processing of the files into the project is finished you will again see "All files loaded" in the status report.

Note, if you start any upload request, and use the one time link to open the Transfer Panel in Globus, but do now actually start a transfer, Materials Commons will wait a long time for that transfer to complete (for two weeks!). So, if you see a report that "Globus is upload files..." that is very old, especially if you do not remember starting a request, and especially if you remember clicking on the link but then deciding not to make the transfer, just ignore it. With two weeks, it will go away.

Globus Download Command

In the project page, you can use the Globus Download Command to download files from your project. Simple click the command and you will see a panel with a link to a Globus Transfer Page.

Unlike the upload command this is not a one-time link. However, it is a snapshot. You can use the link as often as you like, but if files (directories) in your project are added, deleted, or uploaded, those changes will not be reflected in this download link. You can update the link to changes in your project but re-issuing the Globus Download Command. Clicking on the link takes you to a Globus Transfer page:

In this case, since you are transferring from materials commons, the source side of the split panel is filled out and you fill in the destination side with the location of the directory that you want to transfer into (on your Globus Connect Personal endpoint). And start the transfer.

Once Material Commons has setup the download endpoint for you to transfer from, no further processing is needed on the Materials Commons server, so the only progress, status, information you need is the information available from the globus activity panel, and the task reports therein.