From 73eb1f84db28da4e094df88915102dacc6eba1bd Mon Sep 17 00:00:00 2001 From: Jan Siwiec <jan.siwiec@vsb.cz> Date: Sun, 7 Mar 2021 16:30:06 +0100 Subject: [PATCH] Update project-storage.md --- docs.it4i/storage/project-storage.md | 92 +++++++++++++++++++--------- 1 file changed, 64 insertions(+), 28 deletions(-) diff --git a/docs.it4i/storage/project-storage.md b/docs.it4i/storage/project-storage.md index 7ea339800..a45962452 100644 --- a/docs.it4i/storage/project-storage.md +++ b/docs.it4i/storage/project-storage.md @@ -7,52 +7,85 @@ The storage can also be used for computing jobs with low I/O demands or big data ## Technical Overview PROJECT consists of three identical interconnected file storage servers (PROJ1-3) -with HA NFS access and the total capacity of 15PB (3x5PB). +with HA NFS over GPFS access and the total capacity of 15PB. **screenshot from DTS (draw.io?)** -| Specification | | -| ----------------- | -------------------------------------------------- | -| Protocol | NFS over GPFS | -| Total capacity | 15PB | -| Throughput | 12 GB/s (3x4)| -| IO Performance || +| Specification | Total | Per Block | +| ----------------- | ---------------------------------------- | +| Protocol | NFS over GPFS | +| Total capacity | 15PB | 5PB | +| Throughput | 12 GB/s (3x4) | 4GB/s | +| IO Performance |?|?| ## Accessing PROJECT -usage per project, life cycle of the users' folder: +Data stored on the PROJECT storage is bound to a project. +Once the project ends, new data related to this project cannot be written on the PROJECT storage. +However, the data can be migrated to another active project's directory. +Each block of the PROJECT File storage is an independent fileset which allows copying or moving files. -data is bound to project, project disabled = data cannot be written = deleted (details TBD) -data can be trasferred to another (active) project (folder) - -HOWTO: -independent filesets = copy or move allowed +!!!note + Note that 'mv' between independent filesystems is not an atomic operations. ### POSIX File Access -Mountpoint - /mnt/proj1...3 +PROJECT ins an external filesystem accessible from mountpoint `/mnt/proj1` to `proj3`. + +PROJECT can be accessed via the following nodes: + +| Cluster | Node(s) | +| ------------- | ----------------------------- | +| Salomon | Login | +| Barbora | Login, Compute, Vizualization | +| DGX2 | Compute | -accessibility: -salomon: login -barbora: login, cn and viz -(karolina: login, cn, and viz dgx2: cn) +To show path to a directory on the PROJECT storage, use the `it4i-get-project-dir` command: -**script example?** +```console +$ it4i-get-project-dir OPEN-XX-XX +$ /mnt/proj3/OPEN-XX-XX +``` ### User Quotas -monitoring for users (location of project, disk usage) -quota for capacity and inodes per project -ask IT4I for more storage capacity +The PROJECT storage follows general user quotas for capacity and inodes per project set by IT4Innovations. +monitoring for users (location of project, disk usage?) + +If your project requires more storage capacity, +request additional storage space allocation in you application for computational resources (this is the preffered option). + +Alternativly, if the project is already active, contact [IT4I Support][a]. -preferred: ask before starting the project -alternative when project is already active: ask support +### Snapshot Functionality Description -user/project quota +Snapshots are created for all projects on fileset-level by running a cron script on GPFS servers. +Snapshot is created every day and snaphots older thain seven days are deleted. -snapshot functionality description -see info in chat +Snapshot Format: + +```console +/gpfs + | + - exports + | + - DD-XX-XX + | + - .snapshots + | + - YYYY-MM-DD-hhmmss + - YYYY-MM-DD-hhmmss + + + + - OPEN-XX-XX + | + - .snapshots + | + - YYYY-MM-DD-hhmmss + - YYYY-MM-DD-hhmmss +``` ### ACL and Linux Rights nfs4 acl @@ -75,8 +108,11 @@ project is acessible from CN (performance is limited) continuous usage = negati ## Technical Specification -(detailed - separate .md file) +For a detailed technical specification, see the Technical Specification secion. ## Summary barbora table format + +[a]: mailto:support@it4i.cz + -- GitLab