|
*HEAppE Middleware* provides simple and intuitive access to a super-computing infrastructure. This framework unifies the access to different HPC systems through a simple object-oriented client-server interface using standard web services. It provides necessary functions for job management, job monitoring and reporting, user authentication and authorization, file transfer, encryption, and various notification mechanisms. |
|
# HEAppE Middleware
|
|
\ No newline at end of file |
|
*High-End Application Execution Middleware* formerly known as *HPC as a Service Middleware*
|
|
|
|
|
|
|
|
*HPC as a Service* is a well known term in the area of high performance computing. It enables users to access an HPC infrastructure without a need to buy and manage their own physical servers or data center infrastructure. Through this service small and medium enterprises (SMEs) can take advantage of the technology without an upfront investment in the hardware. This approach further lowers the entry barrier for users and SMEs who are interested in utilizing massive parallel computers but often do not have the necessary level of expertise in this area.
|
|
|
|
|
|
|
|
To provide this simple and intuitive access to the supercomputing infrastructure an in-house application framework called *HEAppE* has been developed. This framework is utilizing a mid-layer principle, in software terminology also known as middleware. Middleware manages and provides information about submitted and running jobs and their data between the client application and the HPC infrastructure. *HEAppE* is able to submit required computation or simulation on HPC infrastructure, monitor the progress and notify the user should the need arise. It provides necessary functions for job management, monitoring and reporting, user authentication and authorization, file transfer, encryption, and various notification mechanisms.
|
|
|
|
|
|
|
|
## Architecture
|
|
|
|
|
|
|
|
![architecture](/uploads/f71f4968ad789748e924c4a9b2a41744/architecture.png)
|
|
|
|
|
|
|
|
*HEAppE's* universally designed software architecture enables unified access to~different HPC systems through a simple object-oriented client-server interface using standard web services. Thus providing HPC capabilities to the users but without the necessity to manage the running jobs form the command-line interface of the HPC scheduler directly on the cluster.
|
|
|
|
|
|
|
|
## Web Services
|
|
|
|
|
|
|
|
### UserAndLimitationManagementWs
|
|
|
|
|
|
|
|
* **AuthenticateUserDigitalSignature** – the user authentication using public/private key pair
|
|
|
|
* **AuthenticateUserPassword** – the user authentication via username and password
|
|
|
|
* **GetCurrentUsageAndLimitationsForCurrentUser** – the current usage and constraints for the user
|
|
|
|
|
|
|
|
### ClusterInformationWs
|
|
|
|
|
|
|
|
* **GetCurrentClusterNodeUsage** – information about the current node usage
|
|
|
|
* **ListAvailableClusters** – the list of available clusters
|
|
|
|
|
|
|
|
### JobManagementWs
|
|
|
|
|
|
|
|
* **GetCurrentInfoForJob** – returns base information describing the current task
|
|
|
|
* **ListJobsForCurrentUser** – returns a list of basic information describing all user jobs
|
|
|
|
* **CreateJob** – creates a new job
|
|
|
|
* **SubmitJob** – submits a job to a HPC scheduler queue
|
|
|
|
* **CancelJob** – cancels a running job
|
|
|
|
* **DeleteJob** - deletes job files
|
|
|
|
|
|
|
|
### FileTransferWs
|
|
|
|
|
|
|
|
* **DownloadFileFromCluster** – downloads entire files from the cluster storage
|
|
|
|
* **DownloadPartsOfJobFilesFromCluster** – downloads changes in the file based on the offset
|
|
|
|
* **EndFileTransfer** – ends the file transfer
|
|
|
|
* **GetFileTransferMethod** – acquires object used for data transfer
|
|
|
|
* **ListChangedFilesForJob** – lists newly created or edited files and folders in the base directory
|
|
|
|
|
|
|
|
### JobReportingWs
|
|
|
|
|
|
|
|
* **GetResourceUsageReportForJob** – the used core-hours for the job
|
|
|
|
* **GetUserGroupResourceUsageReport** – the used core-hours for a user group
|
|
|
|
* **GetUserResourceUsageReport** – the used core-hours for the user
|
|
|
|
* **ListAdaptorUserGroups** – a list of all users in the specified group
|
|
|
|
|
|
|
|
## Command Template Prepareation
|
|
|
|
|
|
|
|
TODO
|
|
|
|
|
|
|
|
## Workflow
|
|
|
|
|
|
|
|
![workflow](/uploads/758f42935a4e6fc972c3e7debc93bda5/workflow.png)
|
|
|
|
|
|
|
|
## HEAppE Integration Example (C#)
|
|
|
|
|
|
|
|
TODO |
|
|
|
\ No newline at end of file |