Skip to content
Snippets Groups Projects
Commit bd89c66c authored by David Hrbáč's avatar David Hrbáč
Browse files

Merge branch 'master' into 'virtual_environment' 

# Conflicts:
#   mkdocs.yml
parents be396d05 adc7f60e
No related branches found
No related tags found
5 merge requests!368Update prace.md to document the change from qprace to qprod as the default...,!367Update prace.md to document the change from qprace to qprod as the default...,!366Update prace.md to document the change from qprace to qprod as the default...,!323extended-acls-storage-section,!219Virtual environment, upgrade MKdocs, upgrade Material design
Hide whitespace changes
Inline Side-by-side
Showing
with 1 addition and 893 deletions
docs.it4i/EasyBuild-Hands-on-2018/Introduction.md deleted 100644 → 0
+ 0
− 35
View file @ be396d05
# Introduction
Please, use **only** training account (DD-18-36-\*)
## Setting Up for Building
Set up an environment to use your own modules. ([EasyBuild#modulepath](https://docs.it4i.cz/software/tools/easybuild/#modulepath))
### Variant A
For temporary setup.
```console
module use $HOME/.local/easybuild/modules/all/
```
### Variant B
Modify your .bash_profile for permanent setup and reload environment (logout, login)
```console
cat ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
module use $HOME/.local/easybuild/modules/all/
PATH=$PATH:$HOME/bin
export PATH
```
docs.it4i/EasyBuild-Hands-on-2018/create.md deleted 100644 → 0
+ 0
− 152
View file @ be396d05
# Creating a New Easyconfig From Scratch
Create easyconfig for `moon-buggy` software. The manual procedure is listed below.
Software: `moon-buggy`
Homepage: [https://github.com/seehuhn/moon-buggy](https://github.com/seehuhn/moon-buggy)
**Description**
Moon-buggy is a simple character graphics game, where you drive some kind of car across the moon's surface. Unfortunately there are dangerous craters there. Fortunately your car can jump over them!
## Manual Installation
```console
$ wget https://code.it4i.cz/kru0052/moon-buggy/-/archive/1.0/moon-buggy-1.0.tar.gz
$ tar xvf moon-buggy-1.0.tar.gz
$ cd moon-buggy-1.0
$ ./autogen.sh
$ ./configure --prefix=/home/kru0052/game/moon-buggy-build
$ make
$ ls
acinclude.m4 buggy.h config.h.in copying.h error.c highscore.o keyboard.c main.o meteor.o moon-buggy.info pager.o README terminal.c title.eps xmalloc.o
aclocal.m4 buggy.o config.log cursor.c error.o hpath.c keyboard.o Makefile missing moon-buggy.lsm persona.c realname.c terminal.o title.o xstrdup.c
ANNOUNCE car.img config.status cursor.o game.c hpath.o laser.c Makefile.am mode.c moon-buggy.png persona.o realname.o test-score-modes TODO xstrdup.o
AUTHORS ChangeLog config.sub darray.h game.o img.sed laser.o Makefile.in mode.o moon-buggy.texi queue.c signal.c texinfo.tex vclock.c
autogen.sh checklist configure date.c ground.c INSTALL level.c manpage.in moon-buggy moon-buggy.xpm queue.o signal.o text2c.sed vclock.o
autom4te.cache config.guess configure.ac date.o ground.o install-sh level.o mdate-sh moon-buggy.6 NEWS random.c stamp-h1 THANKS version.texi
buggy.c config.h COPYING depcomp highscore.c instcmds main.c meteor.c moon-buggy.h pager.c random.o stamp-vti title.c xmalloc.c
$ ./moon-buggy
```
## Create Easyconfog From Template
* **Task**: *create easyconfig for `moon-buggy` (use template)* ([Download template.eb](template.eb))
```console
$ cp template.eb moon-buggy-1.0.eb
```
* **Task**: *EASYBLOCK* ... choose easyblock (Analyse manual instalation - only step CONFIGURE AND MAKE -> choose easyblock `ConfigureMake`)
```python
easyblock = 'ConfigureMake'
```
* **Task**: *NAME* ... defined name of the software
```python
name = 'moon-buggy'
```
* **Task**: *VERSION* ... defined versions of the software
```python
version = "1.0"
```
* **Task**: *VERSIONSUFFIX* ... add your login
```python
versionsuffix = "-kru0052"
```
* **Task**: *HOMEPAGE* ... homepage url
```python
homepage = 'https://github.com/seehuhn/moon-buggy'
```
* **Task**: *DESCRIPTION* ... basic software information
```python
description = """Moon-buggy is a simple character graphics game, where you drive some
kind of car across the moon's surface. Unfortunately there are
dangerous craters there. Fortunately your car can jump over them!"""
```
* **Task**: *TOOLCHAIN* ... choose toolchain
```python
toolchain = {'name': 'dummy', 'version': ''}
```
* **Task**: *SOURCE_URLS* ... source urls
```python
source_urls = ['https://code.it4i.cz/kru0052/moon-buggy/-/archive/%(version)s/']
```
* **Task**: *SOURCES* ... package name definition
```python
sources = ['%(name)s-%(version)s.tar.gz']
```
* **Task**: *PRECONFIGOPTS* ... autogen.sh
```python
preconfigopts = "./autogen.sh && "
```
* **Task**: *BUILDDEPENDENCY*
```python
('Autoconf', '2.69')
```
* **Task**: *DEPENDENCY*
```python
('ncurses', '6.1'),
```
* **Task**: *SANITY_CHECK_PATH* ... you must check exists binary file
```python
sanity_check_paths = {
'files': ['bin/moon-buggy', 'com/moon-buggy/mbscore'],
'dirs': ['bin', 'com', 'share'],
}
```
* **Task**: *MODULECLASS* ... choose class
```python
moduleclass = 'tools'
```
* **Task**: *install `moon-buggy` from easyconfig*
```console
$ eb moon-buggy-1.0.eb -r
== temporary log file in case of crash /tmp/eb-ctAvZY/easybuild-GQkRPM.log
== resolving dependencies ...
== processing EasyBuild easyconfig /home/kru0052/game/moon-buggy-1.0.eb
== building and installing moon-buggy/1.0-kru0052...
== fetching files...
...
...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /home/kru0052/.local/easybuild/software/moon-buggy/1.0-kru0052/easybuild/easybuild-moon-buggy-1.0-20181016.094918.log
== Build succeeded for 1 out of 1
== Temporary log file(s) /tmp/eb-ctAvZY/easybuild-GQkRPM.log* have been removed.
== Temporary directory /tmp/eb-ctAvZY has been removed.
```
* **Task**: *load module and run `moon-buggy`*
```console
$ ml moon-buggy/1.0-kru0052
$ moon-buggy
```
docs.it4i/EasyBuild-Hands-on-2018/easybuildandsingularity.md deleted 100644 → 0
+ 0
− 320
View file @ be396d05
# EasyBuild and Singularity
[EasyBuild](https://docs.it4i.cz/software/easybuild/)
[EasyBuild-Singularity](https://docs.it4i.cz/software/tools/easybuild-images/)
* **Tasks**: *create git-2.19.1.eb bootstrap for build singularity image*
```console
$ eb git-2.19.1.eb -C --container-base shub:shahzebsiddiqui/eb-singularity:centos-7.4.1708 --experimental
== temporary log file in case of crash /tmp/eb-E5i5Xx/easybuild-pNROsc.log
== Singularity definition file created at /home/kru0052/.local/easybuild/containers/Singularity.git-2.19.1-Py-3.6-kru0052
== Temporary log file(s) /tmp/eb-E5i5Xx/easybuild-pNROsc.log* have been removed.
== Temporary directory /tmp/eb-E5i5Xx has been removed.
[kru0052@login4.salomon game]$ cat /home/kru0052/.local/easybuild/containers/Singularity.git-2.19.1-Py-3.6-kru0052
Bootstrap: shub
From: shahzebsiddiqui/eb-singularity:centos-7.4.1708
%post
# upgrade easybuild package automatically to latest version
pip install -U easybuild
# change to 'easybuild' user
su - easybuild
eb git-2.19.1.eb --robot --installpath=/app/ --prefix=/scratch --tmpdir=/scratch/tmp
# exit from 'easybuild' user
exit
# cleanup
rm -rf /scratch/tmp/* /scratch/build /scratch/sources /scratch/ebfiles_repo
%runscript
eval "$@"
%environment
source /etc/profile
module use /app/modules/all
module load git/2.19.1-Py-3.6-kru0052
%labels
```
## Use Bootstrap for Installing Singularity Image
Remember, you must sudo **privilege**. (root password)
```console
local $ sudo singularity build image.simg simg
[sudo] password for kru0052:
Using container recipe deffile: sing
Sanitizing environment
Adding base Singularity environment to container
Progress |===================================| 100.0%
Exporting contents of shub://shahzebsiddiqui/eb-singularity:centos-7.4.1708 to /tmp/.singularity-build.rRr52f
User defined %runscript found! Taking priority.
Adding environment to container
Running post scriptlet
+ pip install -U easybuild
Collecting easybuild
Downloading https://files.pythonhosted.org/packages/db/dd/85e9eec7b3c92a7e3ba214f354c03c519ec90dcb6ac7be288dfdd426ddfd/easybuild-3.7.1.tar.gz
Collecting easybuild-easyconfigs==3.7.1 (from easybuild)
Downloading https://files.pythonhosted.org/packages/73/63/b22ff96b8c3e09e04466951c0c3aa7b2230a522792dd3ae37c5fce4c68ea/easybuild-easyconfigs-3.7.1.tar.gz (3.3MB)
100% |################################| 3.3MB 341kB/s
Collecting easybuild-easyblocks==3.7.1 (from easybuild)
Downloading https://files.pythonhosted.org/packages/50/ea/3381a6e85f9a9beee311bed81a03c4900dd11c2a25c1e952b76e9a73486b/easybuild-easyblocks-3.7.1.tar.gz (338kB)
100% |################################| 348kB 1.8MB/s
Collecting easybuild-framework==3.7.1 (from easybuild)
Downloading https://files.pythonhosted.org/packages/d0/f1/a3c897ab19ad36a9a259adc0b31e383a8d322942eda1e59eb4fedee27d09/easybuild-framework-3.7.1.tar.gz (1.7MB)
100% |################################| 1.7MB 669kB/s
Collecting setuptools>=0.6 (from easybuild-easyblocks==3.7.1->easybuild)
Downloading https://files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl (569kB)
100% |################################| 573kB 1.4MB/s
Collecting vsc-install>=0.9.19 (from easybuild-framework==3.7.1->easybuild)
Downloading https://files.pythonhosted.org/packages/b6/03/becd813f5c4e8890254c79db8d2558b658f5a3ab52157bc0c077c6c9beea/vsc-install-0.11.2.tar.gz (61kB)
100% |################################| 71kB 3.1MB/s
Collecting vsc-base>=2.5.8 (from easybuild-framework==3.7.1->easybuild)
Downloading https://files.pythonhosted.org/packages/62/e5/589612e47255627e4752d99018ae7cff8f49ab0fa6b4ba7b2226a76a05d3/vsc-base-2.8.3.tar.gz (104kB)
100% |################################| 112kB 1.8MB/s
Installing collected packages: setuptools, vsc-install, vsc-base, easybuild-framework, easybuild-easyblocks, easybuild-easyconfigs, easybuild
Found existing installation: setuptools 0.9.8
Uninstalling setuptools-0.9.8:
Successfully uninstalled setuptools-0.9.8
Found existing installation: vsc-install 0.10.27
Uninstalling vsc-install-0.10.27:
Successfully uninstalled vsc-install-0.10.27
Running setup.py install for vsc-install ... done
Found existing installation: vsc-base 2.5.8
Uninstalling vsc-base-2.5.8:
Successfully uninstalled vsc-base-2.5.8
Running setup.py install for vsc-base ... done
Found existing installation: easybuild-framework 3.5.1
Uninstalling easybuild-framework-3.5.1:
Successfully uninstalled easybuild-framework-3.5.1
Running setup.py install for easybuild-framework ... done
Found existing installation: easybuild-easyblocks 3.5.1
Uninstalling easybuild-easyblocks-3.5.1:
Successfully uninstalled easybuild-easyblocks-3.5.1
Running setup.py install for easybuild-easyblocks ... done
Found existing installation: easybuild-easyconfigs 3.5.1
Uninstalling easybuild-easyconfigs-3.5.1:
Successfully uninstalled easybuild-easyconfigs-3.5.1
Running setup.py install for easybuild-easyconfigs ... done
Found existing installation: easybuild 3.5.1
Uninstalling easybuild-3.5.1:
Successfully uninstalled easybuild-3.5.1
Running setup.py install for easybuild ... done
Successfully installed easybuild-3.7.1 easybuild-easyblocks-3.7.1 easybuild-easyconfigs-3.7.1 easybuild-framework-3.7.1 setuptools-40.4.3 vsc-base-2.8.3 vsc-install-0.11.2
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
+ su - easybuild
== temporary log file in case of crash /scratch/tmp/eb-NWxNRI/easybuild-EMp20L.log
ERROR: Can't find path /home/easybuild/git-2.19.1.eb
ABORT: Aborting with RETVAL=255
Cleaning up...
```
## Problem Solving
Download [git-2.19.1.eb](git-2.19.1.eb) to local computer.
Change git-2.19.1.eb to /tmp/git-2.19.1.eb into bootstrap.
```console
eb /tmp/git-2.19.1.eb --robot --installpath=/app/ --prefix=/scratch --tmpdir=/scratch/tmp
```
```console
local $ sudo singularity build image.simg simg
Using container recipe deffile: sing
Sanitizing environment
Adding base Singularity environment to container
Progress |===================================| 100.0%
Exporting contents of shub://shahzebsiddiqui/eb-singularity:centos-7.4.1708 to /tmp/.singularity-build.EBQ7qI
User defined %runscript found! Taking priority.
Adding environment to container
Running post scriptlet
+ pip install -U easybuild
Collecting easybuild
Downloading https://files.pythonhosted.org/packages/db/dd/85e9eec7b3c92a7e3ba214f354c03c519ec90dcb6ac7be288dfdd426ddfd/easybuild-3.7.1.tar.gz
Collecting easybuild-easyconfigs==3.7.1 (from easybuild)
Downloading https://files.pythonhosted.org/packages/73/63/b22ff96b8c3e09e04466951c0c3aa7b2230a522792dd3ae37c5fce4c68ea/easybuild-easyconfigs-3.7.1.tar.gz (3.3MB)
100% |################################| 3.3MB 352kB/s
Collecting easybuild-easyblocks==3.7.1 (from easybuild)
Downloading https://files.pythonhosted.org/packages/50/ea/3381a6e85f9a9beee311bed81a03c4900dd11c2a25c1e952b76e9a73486b/easybuild-easyblocks-3.7.1.tar.gz (338kB)
100% |################################| 348kB 1.9MB/s
Collecting easybuild-framework==3.7.1 (from easybuild)
Downloading https://files.pythonhosted.org/packages/d0/f1/a3c897ab19ad36a9a259adc0b31e383a8d322942eda1e59eb4fedee27d09/easybuild-framework-3.7.1.tar.gz (1.7MB)
100% |################################| 1.7MB 590kB/s
Collecting setuptools>=0.6 (from easybuild-easyblocks==3.7.1->easybuild)
Downloading https://files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl (569kB)
100% |################################| 573kB 1.5MB/s
Collecting vsc-install>=0.9.19 (from easybuild-framework==3.7.1->easybuild)
Downloading https://files.pythonhosted.org/packages/b6/03/becd813f5c4e8890254c79db8d2558b658f5a3ab52157bc0c077c6c9beea/vsc-install-0.11.2.tar.gz (61kB)
100% |################################| 71kB 2.8MB/s
Collecting vsc-base>=2.5.8 (from easybuild-framework==3.7.1->easybuild)
Downloading https://files.pythonhosted.org/packages/62/e5/589612e47255627e4752d99018ae7cff8f49ab0fa6b4ba7b2226a76a05d3/vsc-base-2.8.3.tar.gz (104kB)
100% |################################| 112kB 3.3MB/s
Installing collected packages: setuptools, vsc-install, vsc-base, easybuild-framework, easybuild-easyblocks, easybuild-easyconfigs, easybuild
Found existing installation: setuptools 0.9.8
Uninstalling setuptools-0.9.8:
Successfully uninstalled setuptools-0.9.8
Found existing installation: vsc-install 0.10.27
Uninstalling vsc-install-0.10.27:
Successfully uninstalled vsc-install-0.10.27
Running setup.py install for vsc-install ... done
Found existing installation: vsc-base 2.5.8
Uninstalling vsc-base-2.5.8:
Successfully uninstalled vsc-base-2.5.8
Running setup.py install for vsc-base ... done
Found existing installation: easybuild-framework 3.5.1
Uninstalling easybuild-framework-3.5.1:
Successfully uninstalled easybuild-framework-3.5.1
Running setup.py install for easybuild-framework ... -^done
n Found existing installation: easybuild-easyblocks 3.5.1
Uninstalling easybuild-easyblocks-3.5.1:
Successfully uninstalled easybuild-easyblocks-3.5.1
Running setup.py install for easybuild-easyblocks ... done
Found existing installation: easybuild-easyconfigs 3.5.1
^B Uninstalling easybuild-easyconfigs-3.5.1:
n Successfully uninstalled easybuild-easyconfigs-3.5.1
Running setup.py install for easybuild-easyconfigs ... done
Found existing installation: easybuild 3.5.1
Uninstalling easybuild-3.5.1:
Successfully uninstalled easybuild-3.5.1
Running setup.py install for easybuild ... done
Successfully installed easybuild-3.7.1 easybuild-easyblocks-3.7.1 easybuild-easyconfigs-3.7.1 easybuild-framework-3.7.1 setuptools-40.4.3 vsc-base-2.8.3 vsc-install-0.11.2
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
+ su - easybuild
== temporary log file in case of crash /scratch/tmp/eb-kEW7Dr/easybuild-tuPVTd.log
== resolving dependencies ...
== processing EasyBuild easyconfig /usr/easybuild/easyconfigs/z/zlib/zlib-1.2.11.eb
== building and installing zlib/1.2.11...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /app/software/zlib/1.2.11/easybuild/easybuild-zlib-1.2.11-20181023.104624.log
== processing EasyBuild easyconfig /usr/easybuild/easyconfigs/m/M4/M4-1.4.17.eb
== building and installing M4/1.4.17...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /app/software/M4/1.4.17/easybuild/easybuild-M4-1.4.17-20181023.104650.log
== processing EasyBuild easyconfig /usr/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69.eb
== building and installing Autoconf/2.69...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /app/software/Autoconf/2.69/easybuild/easybuild-Autoconf-2.69-20181023.104655.log
== processing EasyBuild easyconfig /tmp/git-2.19.1.eb
== building and installing git/2.19.1-Py-3.6-kru0052...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== permissions...
== packaging...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /app/software/git/2.19.1-Py-3.6-kru0052/easybuild/easybuild-git-2.19.1-20181023.104742.log
== Build succeeded for 4 out of 4
== Temporary log file(s) /scratch/tmp/eb-kEW7Dr/easybuild-tuPVTd.log* have been removed.
== Temporary directory /scratch/tmp/eb-kEW7Dr has been removed.
+ rm -rf '/scratch/tmp/*' /scratch/build /scratch/sources /scratch/ebfiles_repo
Adding deffile section labels to container
Adding runscript
Found an existing definition file
Adding a bootstrap_history directory
Finalizing Singularity container
Calculating final size for metadata...
Environment variables were added, removed, and/or changed during bootstrap.
Variables unique to original image (shahzebsiddiqui/eb-singularity:centos-7.4.1708)
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Variables unique to new image (/tmp/.singularity-build.EBQ7qI)
BASH_ENV=/usr/share/lmod/lmod/init/bash
LMOD_CMD=/usr/share/lmod/lmod/libexec/lmod
LMOD_COLORIZE=yes
LMOD_DIR=/usr/share/lmod/lmod/libexec/
LMOD_FULL_SETTARG_SUPPORT=no
LMOD_PKG=/usr/share/lmod/lmod
LMOD_PREPEND_BLOCK=normal
LMOD_SETTARG_CMD=:
LMOD_arch=x86_64
LMOD_sys=Linux
MANPATH=
MODULEPATH=/etc/lmod/modules:/usr/share/lmod/lmod/modulefiles/
MODULEPATH_ROOT=/usr/modulefiles
MODULESHOME=/usr/share/lmod/lmod
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
USER=
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
Skipping checks
Building Singularity image...
Singularity container built: image.simg
Cleaning up...
```
## Testing the Assembled Image
```console
local $ git --version
git version 2.17.1
local $ singularity exec image.simg git --version
git version 2.19.1
```
docs.it4i/EasyBuild-Hands-on-2018/firstsoftware.md deleted 100644 → 0
+ 0
− 72
View file @ be396d05
# Building Your First Software Using Easyconfigs
Install git version 2.19.0 from ready-made easyconfig.
* **task**: *install git/2.19.0* ([Download git-2.19.0.eb](git-2.19.0.eb))
## Load Module EasyBuild
The effectively apply the changes to the environment that are specified by a module, use `ml` and specify the name of the module. For example, to set up your environment to use `ml EasyBuild`.
To get an overview of the currently loaded modules, use module list or ml (without specifying extra arguments).
```console
$ ml EasyBuild
$ ml
Currently Loaded Modules:
1) EasyBuild/3.7.1 (S)
Where:
S: Module is Sticky, requires --force to unload or purge
```
## Show Installed Dependencies
You can do a `dry-run` overview by supplying `-D/--dry-run` (typically combined with `--robot`, in the form of `-Dr`).
```console
$ eb git-2.19.0.eb -Dr
== temporary log file in case of crash /tmp/eb-xk3mg4/easybuild-f90lIR.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/apps/easybuild/easyconfigs-it4i
* [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18)
* [ ] $CFGS/a/Autoconf/Autoconf-2.68.eb (module: Autoconf/2.68)
* [ ] $CFGS/g/git/git-2.19.0.eb (module: git/2.19.0)
== Temporary log file(s) /tmp/eb-xk3mg4/easybuild-f90lIR.log* have been removed.
== Temporary directory /tmp/eb-xk3mg4 has been removed.
```
## Install Module and All Dependencies
If we try to build git-2.19.0.eb. To enable dependency resolution, use the `--robot` command line option (or `-r` for short).
```console
$ eb git-2.19.0.eb -r
== temporary log file in case of crash /tmp/eb-WEtJ8t/easybuild-dvHmbd.log
== resolving dependencies ...
== processing EasyBuild easyconfig /apps/easybuild/easyconfigs-it4i/a/Autoconf/Autoconf-2.68.eb
== building and installing Autoconf/2.68...
...
...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /apps/all/Autoconf/2.68/easybuild/easybuild-Autoconf-2.68-20181016.085630.log
== processing EasyBuild easyconfig /apps/easybuild/easyconfigs-it4i/g/git/git-2.19.0.eb
== building and installing git/2.19.0...
== fetching files...
...
...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /apps/all/git/2.19.0/easybuild/easybuild-git-2.19.0-20181016.085647.log
== Build succeeded for 2 out of 2
== Temporary log file(s) /tmp/eb-WEtJ8t/easybuild-dvHmbd.log* have been removed.
== Temporary directory /tmp/eb-WEtJ8t has been removed.
```
## Test Installed Module
```console
$ ml git/2.19.0
$ git --version
git version 2.19.0
```
docs.it4i/EasyBuild-Hands-on-2018/git-2.19.0.eb deleted 100644 → 0
+ 0
− 23
View file @ be396d05
# IT4Innovations 2018
easyblock = 'ConfigureMake'
name = 'git'
version = "2.19.0"
homepage = 'http://git-scm.com/'
description = """Git is a free and open source distributed version control system designed
to handle everything from small to very large projects with speed and efficiency."""
toolchain = {'name': 'dummy', 'version': ''}
source_urls = ['https://www.kernel.org/pub/software/scm/git/']
sources = ['%(name)s-%(version)s.tar.gz']
builddependencies = [
('Autoconf', '2.68', '', True)
]
preconfigopts = 'make configure && '
moduleclass = 'tools'
docs.it4i/EasyBuild-Hands-on-2018/git-2.19.1.eb deleted 100644 → 0
+ 0
− 37
View file @ be396d05
# IT4Innovations 2018
easyblock = 'ConfigureMake'
name = 'git'
version = "2.19.1"
versionsuffix = "-Py-3.6-foxik"
homepage = 'http://git-scm.com/'
description = """Git is a free and open source distributed version control system designed
to handle everything from small to very large projects with speed and efficiency."""
toolchain = {'name': 'dummy', 'version': ''}
source_urls = ['https://www.kernel.org/pub/software/scm/git/']
sources = ['%(name)s-%(version)s.tar.gz']
builddependencies = [
('Autoconf', '2.69', '', True)
]
dependencies = [
('zlib', '1.2.11', '', True)
]
preconfigopts = 'make configure && '
sanity_check_paths = {
'files': ['bin/git'],
'dirs': ['bin', 'libexec', 'share'],
}
sanity_check_commands = [
('git', '--version')
]
moduleclass = 'tools'
docs.it4i/EasyBuild-Hands-on-2018/modification.md deleted 100644 → 0
+ 0
− 99
View file @ be396d05
# Modification of an Existing Easyconfig
The goal is to modify the easyconfig for git/2.19.1 so that it meets the requirements below.
## Change Name Easyconfig
* **Task**: *change name* (name-version-Py-version-login)
```console
$ cp git-2.19.0.eb git-2.19.1-Py-3.6-kru0052.eb
```
## Change Software Version
* **Task**: *change git version from 2.19.0 to 2.19.1*
```python
version = "2.19.1"
```
## Add Module Suffix
* **Task**: *add module suffix (all module name git/2.19.1-Py-version-login)*
```python
versionsuffix = "-Py-3.6-kru0052"
```
## Add Dependencies
* **Task**: *cURL/7.56.1 (dummy)*, *expat/2.2.0 (dummy)*, *add Py/3.6 dependency (dummy)*
```python
dependencies = [
('cURL', '7.56.1', '', True),
('expat', '2.2.0', '', True),
('Py', '3.6', '', True)
]
```
## Add Sanity Check Paths
* **Task**: *sanity_check_paths*
```python
sanity_check_paths = {
'files': ['bin/git'],
'dirs': ['bin', 'libexec', 'share'],
}
```
## Add Sanity Check Commands
* **Task**: *sanity_check_commands*
```python
sanity_check_commands = [
('git', '--version')
]
```
## Install Module
```console
$ eb git-2.19.1.eb -r
Couldn't import dot_parser, loading of dot files will not be possible.
== temporary log file in case of crash /tmp/eb-oYDvfO/easybuild-zCM5rM.log
== resolving dependencies ...
== processing EasyBuild easyconfig /home/kru0052/game/git-2.19.1.eb
== building and installing git/2.19.1-Py-3.6-kru0052...
...
...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /home/kru0052/.local/easybuild/software/git/2.19.1-Py-3.6-foxik/easybuild/easybuild-git-2.19.1-20181016.092517.log
== Build succeeded for 1 out of 1
== Temporary log file(s) /tmp/eb-oYDvfO/easybuild-zCM5rM.log* have been removed.
== Temporary directory /tmp/eb-oYDvfO has been removed.
```
## Test New Module
```console
$ ml av git/
-------------------- /home/kru0052/.local/easybuild/modules/all --------------------
git/2.19.0 git/2.19.1-Py-3.6-kru0052 (D)
-------------------- /apps/modules/tools --------------------
git/2.18.0
Where:
D: Default Module
If you need software that is not listed, request it at support@it4i.cz.
$ ml git/2.19.1-Py-3.6-foxik
$ git --version
git version 2.19.1
```
docs.it4i/EasyBuild-Hands-on-2018/template.eb deleted 100644 → 0
+ 0
− 28
View file @ be396d05
# IT4Innovations 2018
easyblock =
name =
version =
versionsuffix =
homepage =
description = """ """
toolchain = {'name': '', 'version': ''}
source_urls = ['']
sources = ['']
preconfigopts =
builddependencies = []
dependencies = []
sanity_check_paths = {
'files': [],
'dirs': [],
}
moduleclass = ''
docs.it4i/EasyBuild-Hands-on-2018/troubleshooting.md deleted 100644 → 0
+ 0
− 126
View file @ be396d05
# Troubleshooting - Typical Problems
## INSTALL - Already Installed
Install module from easyconfig `git-2.15.0-GCC-7.1.0-2.28-Python-3.6.1.eb`
```console
[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb
== temporary log file in case of crash /tmp/eb-KMNHse/easybuild-Jwn_R7.log
== git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK is already installed (module found), skipping
== No easyconfigs left to be built.
== Build succeeded for 0 out of 0
== Temporary log file(s) /tmp/eb-KMNHse/easybuild-Jwn_R7.log* have been removed.
== Temporary directory /tmp/eb-KMNHse has been removed.
```
## REINSTALL - Module Already Loaded
Load module `git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK` and reinstall module
```console
[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -f
== temporary log file in case of crash /tmp/eb-NDp_Rx/easybuild-XgYOq9.log
WARNING: Found one or more non-allowed loaded (EasyBuild-generated) modules in current environment:
* bzip2/1.0.6
* libreadline/6.3
* SQLite/3.13.0
* Tcl/8.6.5
* Tk/8.6.5
* GMP/6.1.1
* XZ/5.2.2
* zlib/1.2.11
* Python/3.6.1
* cURL/7.53.1
* expat/2.2.0
* ncurses/6.0
* gettext/0.19.8.1
* GCCcore/7.1.0
* GCCcore/7.1.0
* binutils/2.28-GCCcore-7.1.0
* Perl/5.26.0-GCC-7.1.0-2.28-bare
* git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK
This is not recommended since it may affect the installation procedure(s) performed by EasyBuild.
To make EasyBuild allow particular loaded modules, use the --allow-loaded-modules configuration option.
To specify action to take when loaded modules are detected, use --detect-loaded-modules={error,ignore,purge,unload,warn}.
See http://easybuild.readthedocs.io/en/latest/Detecting_loaded_modules.html for more information.
== processing EasyBuild easyconfig /home_lustre/kru0052/hands-on/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb
== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK...
== fetching files...
== creating build dir, resetting environment...
== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Module is already loaded (EBROOTGIT is set), installation cannot continue.
== Results of the build can be found in the log file(s) /tmp/eb-NDp_Rx/easybuild-git-2.14.1-20170911.085415.EaWTJ.log
ERROR: Build of /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: 'build failed (first 300 chars): Module is already loaded (EBROOTGIT is set), installation cannot continue.')
```
## INSTALL/REINSTALL - Irresolvable Dependencies
Change `('Python', '3.6.1', '', True)` to `('Python', '3.6.1')` and reinstall module
```console
[kru0052@login2.anselm ~]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -r
== temporary log file in case of crash /tmp/eb-7toWyv/easybuild-koZYcG.log
== processing EasyBuild easyconfig /home_lustre/kru0052/hands-on/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb
== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK...
== fetching files...
== creating build dir, resetting environment...
== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Missing modules for one or more dependencies: Python/3.6.1-GCC-7.1.0-2.28
== Results of the build can be found in the log file(s) /tmp/eb-7toWyv/easybuild-git-2.14.1-20170911.084653.qAHAA.log
ERROR: Build of /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: 'build failed (first 300 chars): Missing modules for one or more dependencies: Python/3.6.1-GCC-7.1.0-2.28')
```
## INSTALL/REINSTALL - Can't Find Path
Use bad easyconfig name `eb git-2.14.1-GCC-7.1.0-2.28.eb`
```console
[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28.eb -f
== temporary log file in case of crash /tmp/eb-keyVcY/easybuild-AgLBmT.log
ERROR: Can't find path /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28.eb
```
## INSTALL/REINSTALL - Sanity Check Failed
Change `'files': ['bin/git']` to `'files': ['bin/git2']`
```console
[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -f
== temporary log file in case of crash /tmp/eb-Lv9b1_/easybuild-h2F2uL.log
== processing EasyBuild easyconfig /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb
== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== postprocessing...
== sanity checking...
== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Sanity check failed: no file of ('bin/git2',) in /home/kru0052/.local/easybuild/software/git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK
== Results of the build can be found in the log file(s) /tmp/eb-Lv9b1_/easybuild-git-2.14.1-20170911.085750.VJyvg.log
ERROR: Build of /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: "build failed (first 300 chars): Sanity check failed: no file of ('bin/git2',) in /home/kru0052/.local/easybuild/software/git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK")
```
## INSTALL/REINSTALL - Couldn't Find File
Change `sources = ['%(name)s-%(version)s.tar.gz']` to `sources = ['%(name)s-%(version)s-test.tar.gz']`
```console
[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -f
== temporary log file in case of crash /tmp/eb-N64arw/easybuild-lXjC9B.log
== processing EasyBuild easyconfig /home_lustre/kru0052/hands-one/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb
== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK...
== fetching files...
== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Couldn't find file git-2.14.1-test.tar.gz anywhere, and downloading it didn't work either... Paths attempted (in order): /home_lustre/kru0052/hands-on/g/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/hands-on/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/hands-on/git-2.14.1-test.tar.gz, /
== Results of the build can be found in the log file(s) /tmp/eb-N64arw/easybuild-git-2.14.1-20170911.085547.BnfZG.log
ERROR: Build of /home_lustre/kru0052/hands-on/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: "build failed (first 300 chars): Couldn't find file git-2.14.1-test.tar.gz anywhere, and downloading it didn't work either... Paths attempted (in order): /home_lustre/kru0052/hands-on/g/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/test-build/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/hands-on/git-2.14.1-test.tar.gz, /")
```
docs.it4i/anselm/job-submission-and-execution.md
+ 1
− 1
View file @ bd89c66c
...@@ -373,7 +373,7 @@ exit ...@@ -373,7 +373,7 @@ exit
In this example, input and executable files are assumed to be preloaded manually in the /scratch/$USER/myjob directory. Note the **mpiprocs** and **ompthreads** qsub options controlling the behavior of the MPI execution. mympiprog.x is executed as one process per node, on all 100 allocated nodes. If mympiprog.x implements OpenMP threads, it will run 16 threads per node. In this example, input and executable files are assumed to be preloaded manually in the /scratch/$USER/myjob directory. Note the **mpiprocs** and **ompthreads** qsub options controlling the behavior of the MPI execution. mympiprog.x is executed as one process per node, on all 100 allocated nodes. If mympiprog.x implements OpenMP threads, it will run 16 threads per node.
More information can be found in the [Running OpenMPI](software/mpi/Running_OpenMPI/) and [Running MPICH2](software/mpi/running-mpich2/) More information can be found in the [Running OpenMPI](../software/mpi/Running_OpenMPI/) and [Running MPICH2](../software/mpi/running-mpich2/)
sections. sections.
### Example Jobscript for Single Node Calculation ### Example Jobscript for Single Node Calculation
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment