access_BomAccessDocumentation_bom dp9 management_user_guide - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki
User Guide
This instruction guides users on how to use the module of nci_admin to check information for a project, such as dp9 on Gadi, including,
- general information on disk and computing resource
- settings for disk space and inodes
- top users in various categories
- overview usage of a project
- history of computing usage
- information for individual user
Load Module nci_admin
You must load the correct module to be able to run any of the following commands to get corresponding information on Gadi,
module purge
module use /projects/access/modules/
module load bom_envs/python_3.7.5
module load nci_admin/3.0.0
Note: The version jumps to 3.0.0 from 1.0.0 to specify that this version of nci_admin is Python3 ready.
Check Individual User
This is to check the detailed information about a core user,
nci_admin.py -p dp9 -u $USER
Here is the output,
Wenming_Lu
[email protected]
wml548
dp9
/g/data/ usage # 2605G share 1.16%
/g/data/ iUsage # 0M share 0.80%
mdss usage # 3855G share 0.22%
/scratch usage # 123929G share 15.13%
Service Unit usage = 2680.00KSU
By default, the script uses the current date to produce the latest information for the user. If aiming for a specific date, an extra argument is needed,
nci_admin.py -p dp9 -u $USER -d 20200401
If you are a guest user, those commands are,
nci_admin.py -p dp9 -u $USER -c guest
nci_admin.py -p dp9 -u $USER -c guest -d 20200401
Note:
- NCI only updates the usage report once a day. Therefore, changes cannot be reflected in real time. Users have to come back the second day to verify the details of changes.
- Uers may get an error if the report(s) for the date input does not exist. This applies to all commands in the later part of the guide.
- NCI only updates mdss report for dp9 monthly.
- Users will get an error message if not a member of dp9 or specifying wrong category of the user, i.e., using -c guest when being a core member and otherwise.
General Project Information
To check the general information of a project,
nci_admin.py -p dp9 -r 1
The output is,
### Project: dp9
massdata Grant=1700T Usage=1154T(67%) Avail= 545T iGrant= 82M iUsage= 0M( 0%) iAvail= 82M
gdata1a Grant= 220T Usage= 167T(76%) Avail= 52T iGrant= 45M iUsage= 28M(62%) iAvail= 16M
scratch Grant= 800T Usage= 335T(41%) Avail= 464T
SU Grant= 9MSU Usage=8.4MSU(93%) Avail=0.6MSU
The total granted quota on disk and inode of mdss, /g/data, /scratch and computing resources in SU are displayed along with the usages. Please note that due to technical reasons, it is not possible to present the information for mdss inode usage.
If you are aiming for a specific date, then likewise,
nci_admin.py -p dp9 -r 1 -d 20200401
However, please be noted that the report may not be available for the date and an error message will be printed out.
List Top Users
To check top 10 users on mdss tape drive of dp9,
nci_admin.py -p dp9 -r 2a
with output as,
USER USAGE
==================
ttl548 162T
hhz548 152T
ycx548 132T
cit548 115T
zhl548 111T
cnf599 79T
sjr548 72T
mxk548 61T
jrf548 43T
zxs548 37T
==================
These are Top Users in MDSS of dp9 for Core Members.
To check top 10 users on /g/data disk of dp9 (output omitted),
nci_admin.py -p dp9 -r 2b
To check top 10 users on /g/data inode of dp9 (output omitted),
nci_admin.py -p dp9 -r 2c
To check top 10 users on /scratch disk of dp9 (output omitted),
nci_admin.py -p dp9 -r 2d
To check top 10 users on SU inode of dp9 (output omitted),
nci_admin.py -p dp9 -r 2e
Similarly, if you are interested in top users for any particular date, an extra argument is needed,
nci_admin.py -p dp9 -r 2a -d 20200401
nci_admin.py -p dp9 -r 2b -d 20200401
nci_admin.py -p dp9 -r 2c -d 20200401
nci_admin.py -p dp9 -r 2d -d 20200401
nci_admin.py -p dp9 -r 2e -d 20200401
For guest users,
nci_admin.py -p dp9 -r 2a -d 20200401 -c guest
nci_admin.py -p dp9 -r 2b -d 20200401 -c guest
nci_admin.py -p dp9 -r 2c -d 20200401 -c guest
nci_admin.py -p dp9 -r 2d -d 20200401 -c guest
nci_admin.py -p dp9 -r 2e -d 20200401 -c guest
Check Current Project Setting
This is to check the current settings for the project,
nci_admin.py -p dp9 -r 3
The settings are,
### General Setting for dp9
Max Individual Share of mdss Tape = 10% (170T)
Max Individual Share of /g/data Disk = 15% (33T)
Max Individual Share of /g/data inode = 15% (6.75M)
Max Individual Share of /scratch Disk = 30% (240T)
Emergency Level of mdss Tape Usage = 90%
Emergency Level of /g/data Disk Usage = 85%
Emergency Level of /g/data inode Usage = 85%
Emergency Level of /scratch Disk Usage = 90%
Emergency mdss Big User Threshold = 50T
Emergency /g/data Disk Big User Threshold = 5T
Emergency /g/data inode Big User Threshold = 2M
Emergency /scratch Disk Big User Threshold = 200T
Guest /g/data/dp9 Usage <= 500G
Guest /g/data/dp9 inode <= 1M
Guest /mdss dp9 Usage = 0T
Guest /scratch/dp9 Usage <= 5000G
You may go to Policy: dp9 Resource Management for detailed meaning of those settings.
Produce Usage Overview
To get an overview of usage on /g/data/ and mdss,
nci_admin.py -p dp9 -r 4
will print out the usages of all users in all categories,
- 1^st^: User NCI ID
- 2^nd^: User's /g/data disk usage (this column is used to order users in the table)
- 3^rd^: User's /g/data inode usage
- 4^th^: User's **mdss tape usage
- 5^rd^: User's /scratch disk usage
- 6^th^: User's SU usage
Here is a sample output,
<<<<<< Core User Info for dp9 >>>>>>
Core /g/data/dp9 Usage = 163T
Core /g/data/dp9 inode = 27M
Core /mdss dp9 Usage = 1096T
Core /scratch/dp9 Usage # 333T===========================================================================###### USER GDATA_DISK GDATA_INODE MDSS_TAPE SCRATCH SU============================================================================
jtl548 51314G 5475K 35839G 2938G 7KSU
cnf599 31641G 256K 81622G 11366G 12KSU
hhz548 18534G 1633K 156605G 43724G 841KSU
fs5892 12108G 264K 8271G 2G 0KSU
...
iib548 4068G 2803K 3599G 2188G 1240KSU
cit548 3904G 220K 118411G 0G 1KSU
...
...
To get an overview for core users for a specific date,
nci_admin.py -p dp9 -r 4 -d 20200401
Similarly, to get an overview for guest users for the current date and a specific date,
nci_admin.py -p dp9 -r 4 -c guest
nci_admin.py -p dp9 -r 4 -c guest -d 20200401
Produce SU Quarterly History
To get the history of SU usage from the beginning of the quarter to the current date,
nci_admin.py -p dp9 -r 5
YYYYYMMDD USAGE DAILY_AVG QUOTA PROJECTION
------------------------------------------------------------------
20200401 0.00MSU 2.29KSU 6.0MSU 0.2MSU
20200402 0.02MSU 11.54KSU 6.0MSU 1.1MSU
20200403 0.02MSU 7.20KSU 6.0MSU 0.7MSU
...
...
20200621 8.09MSU 98.66KSU 9.0MSU 9.0MSU
20200622 8.24MSU 99.28KSU 9.0MSU 9.0MSU
20200623 8.37MSU 99.64KSU 9.0MSU 9.1MSU
The last four columns denote the total SU used, average daily usage, SU quota and projected usage at the end of the quarter.