High Performance Computing Group
Wendian is Mines' new HPC platform coming on line in the winter of 2019. It contains the latest generation of Intel processors, Nvidia GPUs, and IBM Power nodes. In total, it has 80 regular compute nodes plus 7 nodes with GPUs combined to offer over 350 TFLOPs. It also has 3 administration nodes, 6 file system nodes heading up 1152 Tbytes (raw) storage @ over 10 Gbytes/Sec; Wendian runs the CentOS version 7 of linux. Parallel jobs are managed via the Slurm scheduler. The programming languages of choice include C, C++, Fortran, OpenMP, OpenACC, Cuda and MPI. The overall specifications are:
BlueM is Mines legacy HPC platform but still allows researchers to run large simulations in support of the university's core research areas while operating on the forefront of algorithm development. BlueM is a unique high performance computing system from IBM. The overall specifications are:
BlueM is unique in configuration. It contains two independent compute partitions that share a common file system. The two partitions are built using different architectures. The first partition, known as Mc2 (Energy), runs on an IBM BlueGene Q (BGQ). The second partition, known as Aun (Golden), uses the iDataplex architecture. Each of the architectures is optimized for a particular type of parallel application.
Proposals can be submitted using the following instructions and electronic form. This procedure is to be used by Mines' faculty members to request a specified number of node-hours. Allocations will be made on a semi-annual basis, and allocation awards will be valid for six months from the award date. Proposals will be evaluated based on:
Importance of listing Grants and Publications
Researcher who have set allocations in 2018 will be grandfathered to Wendian. To check your allocation set up date run the command:
on AuN or Mc2. Accounts set up this year will begin with 18*.
You will see in the form below, fields for entering information about your proposals, funded research, and HPC related publications. This is important because this information will be used as part of the justification for future machine and infrastructure updates. This information will be reviewed and additional information many be sought if your entries do not appear complete.
Calculation of node hours to request
Researchers with previous allocations on BlueM can determine there usage by running the following commands on either AuN or Mc2:
/opt/utility/aunhrs ############ /opt/utility/mc2hrs ############
where ############ is the account number. These commands will appear to report account information back to the the beginning of last year but since these accounts have only been active on our current accounting system since the fall only the hours used since August will be reported.
You can see the accounts you are authorized to use by running the following command on either AuN or Mc2:
sacctmgr list association cluster=mc2 user=$LOGNAME format=User,Account%20
This will return a list of your accounts. You can then see the association between your account number and your project title by running the following command, replacing ############ with the values from the previous command.
sacctmgr list account ############ format=Account%15,Desc%80
The per core performance of Wendian nodes will roughly 2x-4x of AuN.
Successful proposals will result in an award of a fixed number of node-hours, and faculty members will be able to track the node-hour usage by their group members through the commands given above. We have been rather lax in the past on enforcing limits to the time granted. Because of various contractual obligations we will need to enforce limits. Users will be allowed to run past their allocated hours but at a low priority.
Wendian Specific Policies
Mc2 and AuN Policies
File system Policies
For questions please contact Dr. Timothy Kaiser at: firstname.lastname@example.org
Note on Proposal date: Researchers my submit proposals after the February 8 date given above. Accounts granted based on proposals submitted after February 8 will expire the same date as proposals submitted before February 8. The expiration date, August 18, 2019 may be slipped to a date in the future.
Fill in all of the information and hit "Submit". All fields not marked optional must be filled. The larget text input boxed will scroll but the can also be expanded. After you hit submit you will see a summary of what you have entered. Please save this for your records.
There are two types of accounts available, normal and experimental Experimental accounts are for those people who are new to HPC and need to gain experience before submitting a request for a normal account. If you had a previous BlueM allocation but you did not use a significant portion of it you are expected to request an experimental allocation. Experimental accounts will receive a fixed allocation and thus applicants do not need to specify a Node-Hours as part of the request.