Centre for eResearch Usage Guidelines
The University provides IT resources to a large and varied group of people. All members of the University community are responsible for using these resources effectively, efficiently and in an ethical manner. IT resources must not be used in any way that interferes with the reasonable use of those resources by other IT users, impacts adversely on the University’s reputation or hinders the University in meeting its legal obligations.
Members of the University community must at all times comply with relevant laws, University statutes, policies and standards. IT users who deal with sensitive data must take particular care to ensure that they comply with all laws and University policies and practices relating to the privacy and security of data.
Some units within the University, such as the Centre for eResearch (CeR), maintain additional IT guidelines. IT users to whom those additional IT guidelines apply must also comply with those requirements.
These guidelines apply to you if you are an IT user of CeR, whether you are at the University or elsewhere, and refer to all CeR IT resources. You may also be subject to the rules and policies of your own institution.
This includes people who have been given access to CeR’s facilities by virtue of a grant of computer time through the New Zealand eScience Infrastructure (NeSI), or through an institutional collaboration with CeR.
These guidelines apply particularly to the access to and use of the high-performance computing and data storage facilities provided and operated by the University of Auckland Centre for eResearch (CeR).
By accessing any CeR IT resources you agree to comply with these guidelines and all University IT policies. If you do not agree, you are not permitted to attempt to access any CeR equipment, and you should notify CeR to arrange for closure of your account.
- Opening a cluster account
- Cluster account security
- What are SSH keys?
- Unacceptable use - examples
- Software licensing
- Queuing systems
- Interactive CeR IT resources
- Storage and disk quotas
- Privacy and security
- Research outcomes and acknowledgements
- Eligibility and account management
- Outages and errors
- Alterations to these guidelines
Opening a cluster account
In order to access or use any CeR IT resources, you must first apply to CeR for a cluster account. During the application process, CeR may ask you to provide details of your current or proposed research, and other relevant information such as your sponsoring organisation, academic department and/or supervisor. We will use this information to associate you with a project and to more accurately assess your research computing needs. Please note that an active cluster account does not create a partnership or an employment or study relationship between you and the University.
Cluster account security
Your cluster account is intended for your sole use. You are responsible for any use of your cluster account, or any actions performed by anyone using your cluster account. You must not provide your cluster account credentials, such as your password or private SSH key, to any other person. CeR staff will never need to request this information. If you become aware that the security of your cluster account has been compromised, you must notify CeR immediately. Any prospective IT users who have joined your group or are collaborating with you on a project must request their own cluster accounts from CeR.
What are SSH keys?
An SSH key pair is an authentication mechanism used by the Secure Shell (SSH) network protocol. It consists of two parts, a private key and a public key. CeR staff may request your public SSH key so that we can set up your cluster account for key-based access. Providing your public SSH key to a CeR staff member does not compromise your cluster account and is not a breach of these guidelines.
By contrast, your private SSH key should be treated like a password and never shared with any other person. Additionally, we recommend protecting your private SSH key with a passphrase known only to yourself.
Unacceptable use - examples
Acceptable use of IT resources is governed by the IT Acceptable Use Policy. CeR has some additional constraints on acceptable use arising from our need to ensure that IT users use CeR IT resources fairly and responsibly.
Examples of unacceptable use of IT would include:
- circumventing the designated job scheduler by running a custom scheduler
- running long computing jobs on a login node
- storing unacceptable material on CeR storage devices
Many software packages have been made available to CeR under academic or other non-commercial licences. Because of the terms of these licences, you must obtain the permission of CeR before running any jobs of a commercial nature.
Additionally, some packages are installed on CeR IT resources but are restricted to IT users who are licensed to run the package or packages concerned. If you wish to access any of these packages, you should discuss your situation with an authorised CeR staff member. CeR may require proof of eligibility before giving you access to a restricted package.
CeR manages the scheduling and running of jobs on CeR IT resources by means of queuing systems. Production jobs should be submitted via an appropriate queuing system at all times. Except with the permission of an authorised CeR staff member, you must not attempt to bypass or circumvent any queuing system or run any job interactively on a CeR IT resource that is not an interactive CeR IT resource.
Interactive CeR IT resources
Some CeR IT resources have been made available to IT users for submission of jobs and other work of an interactive nature. Examples of jobs permitted on these CeR IT resources (“interactive CeR IT resources”) include file management (e.g., moving, renaming or deleting files or directories), text editing, code compilation and the testing and optimisation of code.
You should use interactive CeR IT resources appropriately and with consideration for other IT users. CeR IT resources housed within the high-performance computing cluster are by default not for interactive use.
Storage and disk quotas
CeR provides non-volatile storage in the form of hard disks. Space on hard disks is shared among all IT users, and storage limits are enforced by disk quotas. The default quota is the minimum that may be assigned to each IT user. CeR reserves the right to vary the default quota. You must not attempt to circumvent the quota system.
IT users or groups may request extra storage space from CeR. If this request is granted, it will be in the form of an increase in disk quota. Unless otherwise agreed, any such increase will last for the duration of the relevant project, but subject to the right of CeR to adjust quotas as needed from time to time. CeR does not offer data archiving or long-term backup services. If you require such archive or backup solutions, you will need to make your own arrangements.
Some types of problems can cause data loss or corruption, and though CeR has backup strategies in place, it is possible for individual files to be omitted from a backup and for media failures to occur. We recommend that you keep your own backups of any important or sensitive data.
Privacy and security
CeR uses UNIX user and group privileges to restrict access to particular data to authorised IT users and groups. CeR staff are permitted to access all data stored on any CeR IT resources. You must not attempt to access data on any CeR IT resources that belongs to another IT user or group without the express permission of the data owner or of an authorised CeR staff member.
While we endeavour to run a secure system, we cannot entirely prevent malicious access to, modification of, or deletion of data. You should seek advice from an authorised CeR staff member before placing any private, confidential, professionally privileged or classified data on CeR IT resources and you should adequately protect any such data. The University is not liable for any breach of privacy that may occur as a result of data being placed on IT resources.
Each IT user who has a cluster account will receive a home directory, which only that IT user and authorised CeR staff members may access. You should hold most of your research data stored on a CeR IT resource in one of your project directories. A project directory will be accessible by a team of one or more IT users. Where an IT user is a student at the University of Auckland or another university in New Zealand, CeR will normally give the student’s primary supervisor access to the student’s project directories if the primary supervisor has his or her own cluster account. If you are a student who would like to restrict all or part of your project data, you may use UNIX permissions to prevent others from accessing particular files or directories, or alternatively contact CeR at any time for advice or assistance regarding data security.
Research outcomes and acknowledgements
Because CeR and NeSI are required to show evidence that we contribute to the quality and amount of research done by the University community, we will be in contact with you from time to time to gather information about how access to our facilities has assisted you in your work on your project, as well as giving you opportunities to discuss your upcoming research computing needs and priorities with CeR. You are expected to provide us with information about the research outcomes you have obtained by using our facilities. Research outcomes include, but are not limited to, internal project reports, peer-reviewed journal articles, conference presentations, theses and dissertations, books or book sections, patents, and software releases.
You are also expected to acknowledge CeR and NeSI in publications whenever you publish research carried out using our facilities. We suggest the following acknowledgement text:
The author(s) wish to acknowledge the contribution of the NeSI high-performance computing facilities and the staff at the Centre for eResearch at the University of Auckland. New Zealand’s national facilities are provided by the New Zealand eScience Infrastructure (NeSI) and funded jointly by NeSI’s collaborator institutions and through the Ministry of Business, Innovation and Employment’s Infrastructure programme. URL: http://www.nesi.org.nz
Eligibility and account management
CeR grants access to CeR IT resources to applicants who have an eligible research project at the time of their application. To be eligible, a research project must satisfy one of the following criteria:
- The project stems from a contract between the University of Auckland and one or more other legal entities (a commercial project), or
- The project has been granted access to one or more CeR IT resources by NeSI (a NeSI merit project or NeSI institutional project), or
- The project is neither a commercial project, nor a NeSI merit project, nor a NeSI institutional project, and the principal investigator (see below) is a staff member or student at the University of Auckland, the University of Otago, or Landcare Research New Zealand Ltd. (a collaborator project).
The applicant must also provide a clear statement of the research goals of the project, and must be able to satisfy CeR that the project would benefit from access to CeR IT resources.
CeR reserves the right not to accept a collaborator project. Examples of situations in which CeR might refuse a project are if the applicant has not satisfied CeR that access to CeR IT resources will benefit the project, or if CeR determines that we cannot provide access to enough CeR IT resources to advance the research programme without unacceptable loss of service to other IT users.
Each project will have a designated principal investigator, who will be responsible to CeR for the overall management of the project, such as approving the addition and removal of team members and providing information on the project’s progress to CeR.
At the time that you are granted a cluster account, CeR will either create a new project for you or associate you with an existing project, as appropriate. If a new project is created for you, you will be designated the principal investigator. Note that we will only create new projects for applicants who are eligible to be principal investigators.
If you are the principal investigator on a project, you may, at any time, request that another eligible researcher be made principal investigator in your place, or that the project be closed.
CeR reserves the right to close a collaborator project at any time. Situations in which we might close a project are if we are significantly dissatisfied with the use of CeR IT resources in connection with the project, or if we believe the project’s research outcomes (or lack thereof) reflect poorly on CeR or on the University. It is very unlikely that we would take this step without first endeavouring to contact the principal investigator.
Before asking us to close a project, you must take a copy of all data you would like to keep. Once we have closed the project, data associated with it on any CeR IT resource may be deleted at any time without notice.
Closing a project will not necessarily trigger the closure of cluster accounts of any IT users who are members of the project team. If your cluster account is not associated with any active project, you will be able to access interactive CeR IT resources such as designated login devices, but you will not be able to submit jobs using your cluster account. If you find yourself in this position and would like to resume work using CeR IT resources, please contact CeR with details of your research programme so we can add you to an existing project or create a new project for you.
Only persons who are employees or students of a New Zealand university or Crown Research Institute, or members of active project teams, are eligible for cluster accounts. If CeR becomes aware that you are no longer eligible for a cluster account, your cluster account will remain open a further 90 days for the purpose of ending any running jobs and retrieving your remaining data from CeR IT resources. During this period, you will not be able to submit jobs. After this period, your cluster account may be closed and any remaining data securely deleted, unless prior arrangements have been made with an authorised CeR staff member. We may at our sole discretion make exceptions for individual IT users. You may also request closure of your cluster account at any time, including during the 90-day window described above.
Outages and errors
About once a month, CeR has a planned service outage to perform maintenance or upgrades on CeR IT resources, or to address non-urgent problems. We notify all IT users of these outages in advance by email. Normally, during a planned outage, access to the cluster is disabled, queued jobs will not be started and all running jobs will be terminated. When we terminate jobs, we cannot provide snapshots or checkpoints. It is your responsibility to ensure that data is appropriately saved before the start of any outage.
In addition to planned outages, we may also suffer from unplanned outages from time to time. The University cannot guarantee that access to or use of any CeR IT resources will be uninterrupted or free from error. The University is not liable for any data loss, data corruption, lack of research progress or other adverse consequences that may occur as a result of an outage (whether planned or unplanned).
You must not attempt to use any CeR IT resources to run tasks of a safety-critical nature such as medical services or flight control.
For long-running jobs, we strongly encourage you to checkpoint the job, create a snapshot, or otherwise save the job state, at frequent intervals, even if there are no impending scheduled outages. CeR staff are available to provide help or advice regarding these procedures.
CeR will notify IT users of operational matters from time to time by email. You are responsible for advising CeR of a valid email address where you can be reached, and for regularly checking the mailbox to which emails sent to that address are delivered. If your email address changes for any reason, you must notify CeR staff of your new email address.
In the event of a breach of these guidelines, CeR may impose penalties including one or more of the following:
- Termination without notice of one or more running tasks or jobs
- Temporary or permanent reduction of disk quota
- Temporary or permanent reduction of queue priority
- Suspension of job submission privileges
- Suspension or cancellation of the IT user’s cluster account
Dependent upon the nature of the incident, some breaches of these guidelines or of any of the University policies may also be referred to the University, the user’s sponsoring organisation, or both, or to law enforcement agencies.
Alterations to these guidelines
These guidelines will be reviewed from time to time by CeR in conjunction with NeSI and the Board. All IT users will be notified of changes to these guidelines by email before the changes take effect. Exemptions from parts of these guidelines may be sought from the CeR Director based on the needs of a particular project. To be valid, any such exemption must be granted in writing.
Any dispute arising under these guidelines may be heard in the first instance by the CeR Director. Determinations by the CeR Director may be appealed to the NeSI Director and/or the Board.
The following definitions apply throughout these guidelines:
An authorised CeR staff member is any person who is authorised by the CeR Director to instruct IT users regarding the use of that IT resource or to approve its use for the purpose or operation under discussion
The Board means the University of Auckland eResearch Advisory Board
CeR means the University of Auckland Centre for eResearch
A cluster account is the body of usernames, passwords, rights and privileges that a User has on CeR IT resources
The CeR Director is the person employed by the University to direct the operations and activities of CeR.
CeR IT resources include any IT resources (see below) that are operated or administered by CeR. CeR IT resources include, but are not limited to, the Pan high-performance computing cluster, the GPFS cluster storage system, the Data Fabric, and server machines operated by CeR primarily for supporting these devices.
Interactive CeR IT resources are any CeR IT resources that are made available for IT users to use in an interactive fashion, such that jobs run on such CeR IT resources are immediately executed rather than proceeding through a batch queue system.
IT resources refers to any University owned or operated hardware or software and the data that is used or stored on it. All IT resources covered by these guidelines are University operated, even if they are wholly or partly owned by other people or organisations
IT user is anyone using IT resources
Job is any computer program that is executed on a CeR IT resource
NeSI means the New Zealand eScience Infrastructure
NeSI institutional project is any project that has access to the Pan cluster because its principal investigator’s sponsoring organisation has a current institutional subscription, obtained from NeSI, on the Pan cluster.
NeSI merit project is a project that has received a specific allocation of CPU core hours on the Pan cluster from NeSI, usually by satisfying the criteria for scientific merit set forth in the NeSI Access Policy.
Principal investigator, in respect of a project, is the person responsible for overseeing and directing the project and any use of CeR IT resources connected with the project.
Production job is any job started by an IT user that will, if successful, run for more than a few minutes, and that does not require intervention from, or active monitoring by, a user during execution.
Project is a body of research that requires or would benefit from access to high-performance computing resources, and in respect of which at least one person has applied for access to CeR IT resources.
Sponsoring organisation is the organisation with which that IT user is primarily associated for the purposes of carrying out the research in his or her project. In most cases, your sponsoring organisation will be your employer, or the tertiary institution at which you are studying
SSH key pair is a two-part software token used as an authentication mechanism by the Secure Shell (SSH) network protocol, and one of several authentication mechanisms used by CeR IT resources
Unit is an organisational grouping across the University and includes a faculty, research centre or service division, or UniServices
University means the University of Auckland including its subsidiaries
The University community includes all staff (whether permanent, temporary or part time), honorary staff, students (whether full time or part time), contractors, subcontractors, consultants, alumni, associates, business partners or official visitors or guests of members of the University or UniServices.
University-operated hardware is any hardware administered by University of Auckland personnel as part of their duties as employees, whether or not the hardware is owned by the University of Auckland or also administered by other persons; and also includes any hardware ordinarily housed in a University of Auckland data centre.