diff --git a/docs.it4i/EasyBuild-Hands-on-2018/Introduction.md b/docs.it4i/EasyBuild-Hands-on-2018/Introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..a5d6577a8f6880cde5418064783bc5772999fe9a --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/Introduction.md @@ -0,0 +1,35 @@ +# Introduction + +Please, use **only** training account (DD-18-36-\*) + +## Setting Up for Building + +Set up an environment to use your own modules. ([EasyBuild#modulepath](https://docs.it4i.cz/software/tools/easybuild/#modulepath)) + +### Variant A + +For temporary setup. + +```console +module use $HOME/.local/easybuild/modules/all/ +``` + +### Variant B + +Modify your .bash_profile for permanent setup and reload environment (logout, login) + +```console +cat ~/.bash_profile +# .bash_profile + +# Get the aliases and functions +if [ -f ~/.bashrc ]; then +. ~/.bashrc +fi + +# User specific environment and startup programs +module use $HOME/.local/easybuild/modules/all/ + +PATH=$PATH:$HOME/bin +export PATH +``` diff --git a/docs.it4i/EasyBuild-Hands-on-2018/create.md b/docs.it4i/EasyBuild-Hands-on-2018/create.md new file mode 100644 index 0000000000000000000000000000000000000000..246f4de9b52f0c46e9de1457ed03b77dee02ce96 --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/create.md @@ -0,0 +1,152 @@ +# Creating a New Easyconfig From Scratch + +Create easyconfig for `moon-buggy` software. The manual procedure is listed below. + +Software: `moon-buggy` + +Homepage: [https://github.com/seehuhn/moon-buggy](https://github.com/seehuhn/moon-buggy) + +**Description** +Moon-buggy is a simple character graphics game, where you drive some kind of car across the moon's surface. Unfortunately there are dangerous craters there. Fortunately your car can jump over them! + +## Manual Installation + +```console +$ wget https://code.it4i.cz/kru0052/moon-buggy/-/archive/1.0/moon-buggy-1.0.tar.gz +$ tar xvf moon-buggy-1.0.tar.gz +$ cd moon-buggy-1.0 +$ ./autogen.sh +$ ./configure --prefix=/home/kru0052/game/moon-buggy-build +$ make +$ ls +acinclude.m4 buggy.h config.h.in copying.h error.c highscore.o keyboard.c main.o meteor.o moon-buggy.info pager.o README terminal.c title.eps xmalloc.o +aclocal.m4 buggy.o config.log cursor.c error.o hpath.c keyboard.o Makefile missing moon-buggy.lsm persona.c realname.c terminal.o title.o xstrdup.c +ANNOUNCE car.img config.status cursor.o game.c hpath.o laser.c Makefile.am mode.c moon-buggy.png persona.o realname.o test-score-modes TODO xstrdup.o +AUTHORS ChangeLog config.sub darray.h game.o img.sed laser.o Makefile.in mode.o moon-buggy.texi queue.c signal.c texinfo.tex vclock.c +autogen.sh checklist configure date.c ground.c INSTALL level.c manpage.in moon-buggy moon-buggy.xpm queue.o signal.o text2c.sed vclock.o +autom4te.cache config.guess configure.ac date.o ground.o install-sh level.o mdate-sh moon-buggy.6 NEWS random.c stamp-h1 THANKS version.texi +buggy.c config.h COPYING depcomp highscore.c instcmds main.c meteor.c moon-buggy.h pager.c random.o stamp-vti title.c xmalloc.c +$ ./moon-buggy +``` + +## Create Easyconfog From Template + +* **Task**: *create easyconfig for `moon-buggy` (use template)* ([Download template.eb](template.eb)) + +```console +$ cp template.eb moon-buggy-1.0.eb +``` + +* **Task**: *EASYBLOCK* ... choose easyblock (Analyse manual instalation - only step CONFIGURE AND MAKE -> choose easyblock `ConfigureMake`) + +```python +easyblock = 'ConfigureMake' +``` + +* **Task**: *NAME* ... defined name of the software + +```python +name = 'moon-buggy' +``` + +* **Task**: *VERSION* ... defined versions of the software + +```python +version = "1.0" +``` + +* **Task**: *VERSIONSUFFIX* ... add your login + +```python +versionsuffix = "-kru0052" +``` + +* **Task**: *HOMEPAGE* ... homepage url + +```python +homepage = 'https://github.com/seehuhn/moon-buggy' +``` + +* **Task**: *DESCRIPTION* ... basic software information + +```python +description = """Moon-buggy is a simple character graphics game, where you drive some +kind of car across the moon's surface. Unfortunately there are +dangerous craters there. Fortunately your car can jump over them!""" +``` + +* **Task**: *TOOLCHAIN* ... choose toolchain + +```python +toolchain = {'name': 'dummy', 'version': ''} +``` + +* **Task**: *SOURCE_URLS* ... source urls + +```python +source_urls = ['https://code.it4i.cz/kru0052/moon-buggy/-/archive/%(version)s/'] +``` + +* **Task**: *SOURCES* ... package name definition + +```python +sources = ['%(name)s-%(version)s.tar.gz'] +``` + +* **Task**: *PRECONFIGOPTS* ... autogen.sh + +```python +preconfigopts = "./autogen.sh && " +``` + +* **Task**: *BUILDDEPENDENCY* + +```python +('Autoconf', '2.69') +``` + +* **Task**: *DEPENDENCY* + +```python + ('ncurses', '6.1'), +``` + +* **Task**: *SANITY_CHECK_PATH* ... you must check exists binary file + +```python +sanity_check_paths = { + 'files': ['bin/moon-buggy', 'com/moon-buggy/mbscore'], + 'dirs': ['bin', 'com', 'share'], +} +``` + +* **Task**: *MODULECLASS* ... choose class + +```python +moduleclass = 'tools' +``` + +* **Task**: *install `moon-buggy` from easyconfig* + +```console +$ eb moon-buggy-1.0.eb -r +== temporary log file in case of crash /tmp/eb-ctAvZY/easybuild-GQkRPM.log +== resolving dependencies ... +== processing EasyBuild easyconfig /home/kru0052/game/moon-buggy-1.0.eb +== building and installing moon-buggy/1.0-kru0052... +== fetching files... +... +... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /home/kru0052/.local/easybuild/software/moon-buggy/1.0-kru0052/easybuild/easybuild-moon-buggy-1.0-20181016.094918.log +== Build succeeded for 1 out of 1 +== Temporary log file(s) /tmp/eb-ctAvZY/easybuild-GQkRPM.log* have been removed. +== Temporary directory /tmp/eb-ctAvZY has been removed. +``` + +* **Task**: *load module and run `moon-buggy`* + +```console +$ ml moon-buggy/1.0-kru0052 +$ moon-buggy +``` diff --git a/docs.it4i/EasyBuild-Hands-on-2018/easybuildandsingularity.md b/docs.it4i/EasyBuild-Hands-on-2018/easybuildandsingularity.md new file mode 100644 index 0000000000000000000000000000000000000000..b5611d6c05f7fca8068c19ba7dbbcf78f19100b4 --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/easybuildandsingularity.md @@ -0,0 +1,320 @@ +# EasyBuild and Singularity + +[EasyBuild](https://docs.it4i.cz/software/easybuild/) + +[EasyBuild-Singularity](https://docs.it4i.cz/software/tools/easybuild-images/) + +* **Tasks**: *create git-2.19.1.eb bootstrap for build singularity image* + +```console +$ eb git-2.19.1.eb -C --container-base shub:shahzebsiddiqui/eb-singularity:centos-7.4.1708 --experimental +== temporary log file in case of crash /tmp/eb-E5i5Xx/easybuild-pNROsc.log +== Singularity definition file created at /home/kru0052/.local/easybuild/containers/Singularity.git-2.19.1-Py-3.6-kru0052 +== Temporary log file(s) /tmp/eb-E5i5Xx/easybuild-pNROsc.log* have been removed. +== Temporary directory /tmp/eb-E5i5Xx has been removed. +[kru0052@login4.salomon game]$ cat /home/kru0052/.local/easybuild/containers/Singularity.git-2.19.1-Py-3.6-kru0052 +Bootstrap: shub +From: shahzebsiddiqui/eb-singularity:centos-7.4.1708 + +%post + +# upgrade easybuild package automatically to latest version +pip install -U easybuild + +# change to 'easybuild' user +su - easybuild + +eb git-2.19.1.eb --robot --installpath=/app/ --prefix=/scratch --tmpdir=/scratch/tmp + +# exit from 'easybuild' user +exit + +# cleanup +rm -rf /scratch/tmp/* /scratch/build /scratch/sources /scratch/ebfiles_repo + +%runscript +eval "$@" + +%environment +source /etc/profile +module use /app/modules/all +module load git/2.19.1-Py-3.6-kru0052 + +%labels + +``` + +## Use Bootstrap for Installing Singularity Image + +Remember, you must sudo **privilege**. (root password) + +```console +local $ sudo singularity build image.simg simg +[sudo] password for kru0052: +Using container recipe deffile: sing +Sanitizing environment +Adding base Singularity environment to container +Progress |===================================| 100.0% +Exporting contents of shub://shahzebsiddiqui/eb-singularity:centos-7.4.1708 to /tmp/.singularity-build.rRr52f +User defined %runscript found! Taking priority. +Adding environment to container +Running post scriptlet ++ pip install -U easybuild +Collecting easybuild + Downloading https://files.pythonhosted.org/packages/db/dd/85e9eec7b3c92a7e3ba214f354c03c519ec90dcb6ac7be288dfdd426ddfd/easybuild-3.7.1.tar.gz +Collecting easybuild-easyconfigs==3.7.1 (from easybuild) + Downloading https://files.pythonhosted.org/packages/73/63/b22ff96b8c3e09e04466951c0c3aa7b2230a522792dd3ae37c5fce4c68ea/easybuild-easyconfigs-3.7.1.tar.gz (3.3MB) + 100% |################################| 3.3MB 341kB/s +Collecting easybuild-easyblocks==3.7.1 (from easybuild) + Downloading https://files.pythonhosted.org/packages/50/ea/3381a6e85f9a9beee311bed81a03c4900dd11c2a25c1e952b76e9a73486b/easybuild-easyblocks-3.7.1.tar.gz (338kB) + 100% |################################| 348kB 1.8MB/s +Collecting easybuild-framework==3.7.1 (from easybuild) + Downloading https://files.pythonhosted.org/packages/d0/f1/a3c897ab19ad36a9a259adc0b31e383a8d322942eda1e59eb4fedee27d09/easybuild-framework-3.7.1.tar.gz (1.7MB) + 100% |################################| 1.7MB 669kB/s +Collecting setuptools>=0.6 (from easybuild-easyblocks==3.7.1->easybuild) + Downloading https://files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl (569kB) + 100% |################################| 573kB 1.4MB/s +Collecting vsc-install>=0.9.19 (from easybuild-framework==3.7.1->easybuild) + Downloading https://files.pythonhosted.org/packages/b6/03/becd813f5c4e8890254c79db8d2558b658f5a3ab52157bc0c077c6c9beea/vsc-install-0.11.2.tar.gz (61kB) + 100% |################################| 71kB 3.1MB/s +Collecting vsc-base>=2.5.8 (from easybuild-framework==3.7.1->easybuild) + Downloading https://files.pythonhosted.org/packages/62/e5/589612e47255627e4752d99018ae7cff8f49ab0fa6b4ba7b2226a76a05d3/vsc-base-2.8.3.tar.gz (104kB) + 100% |################################| 112kB 1.8MB/s +Installing collected packages: setuptools, vsc-install, vsc-base, easybuild-framework, easybuild-easyblocks, easybuild-easyconfigs, easybuild + Found existing installation: setuptools 0.9.8 + Uninstalling setuptools-0.9.8: + Successfully uninstalled setuptools-0.9.8 + Found existing installation: vsc-install 0.10.27 + Uninstalling vsc-install-0.10.27: + Successfully uninstalled vsc-install-0.10.27 + Running setup.py install for vsc-install ... done + Found existing installation: vsc-base 2.5.8 + Uninstalling vsc-base-2.5.8: + Successfully uninstalled vsc-base-2.5.8 + Running setup.py install for vsc-base ... done + Found existing installation: easybuild-framework 3.5.1 + Uninstalling easybuild-framework-3.5.1: + Successfully uninstalled easybuild-framework-3.5.1 + Running setup.py install for easybuild-framework ... done + Found existing installation: easybuild-easyblocks 3.5.1 + Uninstalling easybuild-easyblocks-3.5.1: + Successfully uninstalled easybuild-easyblocks-3.5.1 + Running setup.py install for easybuild-easyblocks ... done + Found existing installation: easybuild-easyconfigs 3.5.1 + Uninstalling easybuild-easyconfigs-3.5.1: + Successfully uninstalled easybuild-easyconfigs-3.5.1 + Running setup.py install for easybuild-easyconfigs ... done + Found existing installation: easybuild 3.5.1 + Uninstalling easybuild-3.5.1: + Successfully uninstalled easybuild-3.5.1 + Running setup.py install for easybuild ... done +Successfully installed easybuild-3.7.1 easybuild-easyblocks-3.7.1 easybuild-easyconfigs-3.7.1 easybuild-framework-3.7.1 setuptools-40.4.3 vsc-base-2.8.3 vsc-install-0.11.2 +You are using pip version 9.0.1, however version 18.1 is available. +You should consider upgrading via the 'pip install --upgrade pip' command. ++ su - easybuild +== temporary log file in case of crash /scratch/tmp/eb-NWxNRI/easybuild-EMp20L.log +ERROR: Can't find path /home/easybuild/git-2.19.1.eb +ABORT: Aborting with RETVAL=255 +Cleaning up... +``` + +## Resolve Problem + +Download [git-2.19.1.eb](git-2.19.1.eb) to local computer. + +Change git-2.19.1.eb to /tmp/git-2.19.1.eb into bootstrap. + +```console +eb /tmp/git-2.19.1.eb --robot --installpath=/app/ --prefix=/scratch --tmpdir=/scratch/tmp +``` + +```console +local $ sudo singularity build image.simg simg +Using container recipe deffile: sing +Sanitizing environment +Adding base Singularity environment to container +Progress |===================================| 100.0% +Exporting contents of shub://shahzebsiddiqui/eb-singularity:centos-7.4.1708 to /tmp/.singularity-build.EBQ7qI +User defined %runscript found! Taking priority. +Adding environment to container +Running post scriptlet ++ pip install -U easybuild +Collecting easybuild + Downloading https://files.pythonhosted.org/packages/db/dd/85e9eec7b3c92a7e3ba214f354c03c519ec90dcb6ac7be288dfdd426ddfd/easybuild-3.7.1.tar.gz +Collecting easybuild-easyconfigs==3.7.1 (from easybuild) + Downloading https://files.pythonhosted.org/packages/73/63/b22ff96b8c3e09e04466951c0c3aa7b2230a522792dd3ae37c5fce4c68ea/easybuild-easyconfigs-3.7.1.tar.gz (3.3MB) + 100% |################################| 3.3MB 352kB/s +Collecting easybuild-easyblocks==3.7.1 (from easybuild) + Downloading https://files.pythonhosted.org/packages/50/ea/3381a6e85f9a9beee311bed81a03c4900dd11c2a25c1e952b76e9a73486b/easybuild-easyblocks-3.7.1.tar.gz (338kB) + 100% |################################| 348kB 1.9MB/s +Collecting easybuild-framework==3.7.1 (from easybuild) + Downloading https://files.pythonhosted.org/packages/d0/f1/a3c897ab19ad36a9a259adc0b31e383a8d322942eda1e59eb4fedee27d09/easybuild-framework-3.7.1.tar.gz (1.7MB) + 100% |################################| 1.7MB 590kB/s +Collecting setuptools>=0.6 (from easybuild-easyblocks==3.7.1->easybuild) + Downloading https://files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl (569kB) + 100% |################################| 573kB 1.5MB/s +Collecting vsc-install>=0.9.19 (from easybuild-framework==3.7.1->easybuild) + Downloading https://files.pythonhosted.org/packages/b6/03/becd813f5c4e8890254c79db8d2558b658f5a3ab52157bc0c077c6c9beea/vsc-install-0.11.2.tar.gz (61kB) + 100% |################################| 71kB 2.8MB/s +Collecting vsc-base>=2.5.8 (from easybuild-framework==3.7.1->easybuild) + Downloading https://files.pythonhosted.org/packages/62/e5/589612e47255627e4752d99018ae7cff8f49ab0fa6b4ba7b2226a76a05d3/vsc-base-2.8.3.tar.gz (104kB) + 100% |################################| 112kB 3.3MB/s +Installing collected packages: setuptools, vsc-install, vsc-base, easybuild-framework, easybuild-easyblocks, easybuild-easyconfigs, easybuild + Found existing installation: setuptools 0.9.8 + Uninstalling setuptools-0.9.8: + Successfully uninstalled setuptools-0.9.8 + Found existing installation: vsc-install 0.10.27 + Uninstalling vsc-install-0.10.27: + Successfully uninstalled vsc-install-0.10.27 + Running setup.py install for vsc-install ... done + Found existing installation: vsc-base 2.5.8 + Uninstalling vsc-base-2.5.8: + Successfully uninstalled vsc-base-2.5.8 + Running setup.py install for vsc-base ... done + Found existing installation: easybuild-framework 3.5.1 + Uninstalling easybuild-framework-3.5.1: + Successfully uninstalled easybuild-framework-3.5.1 + Running setup.py install for easybuild-framework ... -^done +n Found existing installation: easybuild-easyblocks 3.5.1 + Uninstalling easybuild-easyblocks-3.5.1: + Successfully uninstalled easybuild-easyblocks-3.5.1 + Running setup.py install for easybuild-easyblocks ... done + Found existing installation: easybuild-easyconfigs 3.5.1 +^B Uninstalling easybuild-easyconfigs-3.5.1: +n Successfully uninstalled easybuild-easyconfigs-3.5.1 + Running setup.py install for easybuild-easyconfigs ... done + Found existing installation: easybuild 3.5.1 + Uninstalling easybuild-3.5.1: + Successfully uninstalled easybuild-3.5.1 + Running setup.py install for easybuild ... done +Successfully installed easybuild-3.7.1 easybuild-easyblocks-3.7.1 easybuild-easyconfigs-3.7.1 easybuild-framework-3.7.1 setuptools-40.4.3 vsc-base-2.8.3 vsc-install-0.11.2 +You are using pip version 9.0.1, however version 18.1 is available. +You should consider upgrading via the 'pip install --upgrade pip' command. ++ su - easybuild +== temporary log file in case of crash /scratch/tmp/eb-kEW7Dr/easybuild-tuPVTd.log +== resolving dependencies ... +== processing EasyBuild easyconfig /usr/easybuild/easyconfigs/z/zlib/zlib-1.2.11.eb +== building and installing zlib/1.2.11... +== fetching files... +== creating build dir, resetting environment... +== unpacking... +== patching... +== preparing... +== configuring... +== building... +== testing... +== installing... +== taking care of extensions... +== postprocessing... +== sanity checking... +== cleaning up... +== creating module... +== permissions... +== packaging... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /app/software/zlib/1.2.11/easybuild/easybuild-zlib-1.2.11-20181023.104624.log +== processing EasyBuild easyconfig /usr/easybuild/easyconfigs/m/M4/M4-1.4.17.eb +== building and installing M4/1.4.17... +== fetching files... +== creating build dir, resetting environment... +== unpacking... +== patching... +== preparing... +== configuring... +== building... +== testing... +== installing... +== taking care of extensions... +== postprocessing... +== sanity checking... +== cleaning up... +== creating module... +== permissions... +== packaging... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /app/software/M4/1.4.17/easybuild/easybuild-M4-1.4.17-20181023.104650.log +== processing EasyBuild easyconfig /usr/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69.eb +== building and installing Autoconf/2.69... +== fetching files... +== creating build dir, resetting environment... +== unpacking... +== patching... +== preparing... +== configuring... +== building... +== testing... +== installing... +== taking care of extensions... +== postprocessing... +== sanity checking... +== cleaning up... +== creating module... +== permissions... +== packaging... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /app/software/Autoconf/2.69/easybuild/easybuild-Autoconf-2.69-20181023.104655.log +== processing EasyBuild easyconfig /tmp/git-2.19.1.eb +== building and installing git/2.19.1-Py-3.6-kru0052... +== fetching files... +== creating build dir, resetting environment... +== unpacking... +== patching... +== preparing... +== configuring... +== building... +== testing... +== installing... +== taking care of extensions... +== postprocessing... +== sanity checking... +== cleaning up... +== creating module... +== permissions... +== packaging... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /app/software/git/2.19.1-Py-3.6-kru0052/easybuild/easybuild-git-2.19.1-20181023.104742.log +== Build succeeded for 4 out of 4 +== Temporary log file(s) /scratch/tmp/eb-kEW7Dr/easybuild-tuPVTd.log* have been removed. +== Temporary directory /scratch/tmp/eb-kEW7Dr has been removed. ++ rm -rf '/scratch/tmp/*' /scratch/build /scratch/sources /scratch/ebfiles_repo +Adding deffile section labels to container +Adding runscript +Found an existing definition file +Adding a bootstrap_history directory +Finalizing Singularity container +Calculating final size for metadata... +Environment variables were added, removed, and/or changed during bootstrap. +Variables unique to original image (shahzebsiddiqui/eb-singularity:centos-7.4.1708) +PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +Variables unique to new image (/tmp/.singularity-build.EBQ7qI) +BASH_ENV=/usr/share/lmod/lmod/init/bash +LMOD_CMD=/usr/share/lmod/lmod/libexec/lmod +LMOD_COLORIZE=yes +LMOD_DIR=/usr/share/lmod/lmod/libexec/ +LMOD_FULL_SETTARG_SUPPORT=no +LMOD_PKG=/usr/share/lmod/lmod +LMOD_PREPEND_BLOCK=normal +LMOD_SETTARG_CMD=: +LMOD_arch=x86_64 +LMOD_sys=Linux +MANPATH= +MODULEPATH=/etc/lmod/modules:/usr/share/lmod/lmod/modulefiles/ +MODULEPATH_ROOT=/usr/modulefiles +MODULESHOME=/usr/share/lmod/lmod +PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin +USER= +XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop +Skipping checks +Building Singularity image... +Singularity container built: image.simg +Cleaning up... +``` + +## Test Builded Image + +```console +local $ git --version +git version 2.17.1 +local $ singularity exec image.simg git --version +git version 2.19.1 +``` diff --git a/docs.it4i/EasyBuild-Hands-on-2018/firstsoftware.md b/docs.it4i/EasyBuild-Hands-on-2018/firstsoftware.md new file mode 100644 index 0000000000000000000000000000000000000000..4605e5ea5ebc383e37faec98dfbedccbc9e22c8e --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/firstsoftware.md @@ -0,0 +1,72 @@ +# Building Your First Software Using Easyconfigs + +Install git version 2.19.0 from ready-made easyconfig. + +* **task**: *install git/2.19.0* ([Download git-2.19.0.eb](git-2.19.0.eb)) + +## Load Module EasyBuild + +The effectively apply the changes to the environment that are specified by a module, use `ml` and specify the name of the module. For example, to set up your environment to use `ml EasyBuild`. + +To get an overview of the currently loaded modules, use module list or ml (without specifying extra arguments). + +```console +$ ml EasyBuild +$ ml + +Currently Loaded Modules: + 1) EasyBuild/3.7.1 (S) + + Where: + S: Module is Sticky, requires --force to unload or purge +``` + +## Show Installed Dependencies + +You can do a `dry-run` overview by supplying `-D/--dry-run` (typically combined with `--robot`, in the form of `-Dr`). + +```console +$ eb git-2.19.0.eb -Dr +== temporary log file in case of crash /tmp/eb-xk3mg4/easybuild-f90lIR.log +Dry run: printing build status of easyconfigs and dependencies +CFGS=/apps/easybuild/easyconfigs-it4i + * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18) + * [ ] $CFGS/a/Autoconf/Autoconf-2.68.eb (module: Autoconf/2.68) + * [ ] $CFGS/g/git/git-2.19.0.eb (module: git/2.19.0) +== Temporary log file(s) /tmp/eb-xk3mg4/easybuild-f90lIR.log* have been removed. +== Temporary directory /tmp/eb-xk3mg4 has been removed. +``` + +## Install Module and All Dependencies + +If we try to build git-2.19.0.eb. To enable dependency resolution, use the `--robot` command line option (or `-r` for short). + +```console +$ eb git-2.19.0.eb -r +== temporary log file in case of crash /tmp/eb-WEtJ8t/easybuild-dvHmbd.log +== resolving dependencies ... +== processing EasyBuild easyconfig /apps/easybuild/easyconfigs-it4i/a/Autoconf/Autoconf-2.68.eb +== building and installing Autoconf/2.68... +... +... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /apps/all/Autoconf/2.68/easybuild/easybuild-Autoconf-2.68-20181016.085630.log +== processing EasyBuild easyconfig /apps/easybuild/easyconfigs-it4i/g/git/git-2.19.0.eb +== building and installing git/2.19.0... +== fetching files... +... +... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /apps/all/git/2.19.0/easybuild/easybuild-git-2.19.0-20181016.085647.log +== Build succeeded for 2 out of 2 +== Temporary log file(s) /tmp/eb-WEtJ8t/easybuild-dvHmbd.log* have been removed. +== Temporary directory /tmp/eb-WEtJ8t has been removed. +``` + +## Test Installed Module + +```console +$ ml git/2.19.0 +$ git --version +git version 2.19.0 +``` diff --git a/docs.it4i/EasyBuild-Hands-on-2018/git-2.19.0.eb b/docs.it4i/EasyBuild-Hands-on-2018/git-2.19.0.eb new file mode 100644 index 0000000000000000000000000000000000000000..f5cebb13482d1c6f4ef6ab31bd632ab84b60a76b --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/git-2.19.0.eb @@ -0,0 +1,23 @@ +# IT4Innovations 2018 + +easyblock = 'ConfigureMake' + +name = 'git' +version = "2.19.0" + +homepage = 'http://git-scm.com/' +description = """Git is a free and open source distributed version control system designed +to handle everything from small to very large projects with speed and efficiency.""" + +toolchain = {'name': 'dummy', 'version': ''} + +source_urls = ['https://www.kernel.org/pub/software/scm/git/'] +sources = ['%(name)s-%(version)s.tar.gz'] + +builddependencies = [ + ('Autoconf', '2.68', '', True) +] + +preconfigopts = 'make configure && ' + +moduleclass = 'tools' diff --git a/docs.it4i/EasyBuild-Hands-on-2018/git-2.19.1.eb b/docs.it4i/EasyBuild-Hands-on-2018/git-2.19.1.eb new file mode 100644 index 0000000000000000000000000000000000000000..66077615669048b6e7931e042d5074fa7c6e13a0 --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/git-2.19.1.eb @@ -0,0 +1,37 @@ +# IT4Innovations 2018 + +easyblock = 'ConfigureMake' + +name = 'git' +version = "2.19.1" +versionsuffix = "-Py-3.6-foxik" + +homepage = 'http://git-scm.com/' +description = """Git is a free and open source distributed version control system designed +to handle everything from small to very large projects with speed and efficiency.""" + +toolchain = {'name': 'dummy', 'version': ''} + +source_urls = ['https://www.kernel.org/pub/software/scm/git/'] +sources = ['%(name)s-%(version)s.tar.gz'] + +builddependencies = [ + ('Autoconf', '2.69', '', True) +] + +dependencies = [ + ('zlib', '1.2.11', '', True) +] + +preconfigopts = 'make configure && ' + +sanity_check_paths = { + 'files': ['bin/git'], + 'dirs': ['bin', 'libexec', 'share'], +} + +sanity_check_commands = [ + ('git', '--version') +] + +moduleclass = 'tools' diff --git a/docs.it4i/EasyBuild-Hands-on-2018/modification.md b/docs.it4i/EasyBuild-Hands-on-2018/modification.md new file mode 100644 index 0000000000000000000000000000000000000000..32d6a2b4ac88bf45d71bf29cb69b0b760768da3e --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/modification.md @@ -0,0 +1,99 @@ +# Modification of an Existing Easyconfig + +The goal is to modify the easyconfig for git/2.19.1 so that it meets the requirements below. + +## Change Name Easyconfig + +* **Task**: *change name* (name-version-Py-version-login) + +```console +$ cp git-2.19.0.eb git-2.19.1-Py-3.6-kru0052.eb +``` + +## Change Software Version + +* **Task**: *change git version from 2.19.0 to 2.19.1* + +```python +version = "2.19.1" +``` + +## Add Module Suffix + +* **Task**: *add module suffix (all module name git/2.19.1-Py-version-login)* + +```python +versionsuffix = "-Py-3.6-kru0052" +``` + +## Add Dependencies + +* **Task**: *cURL/7.56.1 (dummy)*, *expat/2.2.0 (dummy)*, *add Py/3.6 dependency (dummy)* + +```python +dependencies = [ + ('cURL', '7.56.1', '', True), + ('expat', '2.2.0', '', True), + ('Py', '3.6', '', True) +] +``` + +## Add Sanity Check Paths + +* **Task**: *sanity_check_paths* + +```python +sanity_check_paths = { + 'files': ['bin/git'], + 'dirs': ['bin', 'libexec', 'share'], +} +``` + +## Add Sanity Check Commands + +* **Task**: *sanity_check_commands* + +```python +sanity_check_commands = [ + ('git', '--version') +] +``` + +## Install Module + +```console +$ eb git-2.19.1.eb -r +Couldn't import dot_parser, loading of dot files will not be possible. +== temporary log file in case of crash /tmp/eb-oYDvfO/easybuild-zCM5rM.log +== resolving dependencies ... +== processing EasyBuild easyconfig /home/kru0052/game/git-2.19.1.eb +== building and installing git/2.19.1-Py-3.6-kru0052... +... +... +== COMPLETED: Installation ended successfully +== Results of the build can be found in the log file(s) /home/kru0052/.local/easybuild/software/git/2.19.1-Py-3.6-foxik/easybuild/easybuild-git-2.19.1-20181016.092517.log +== Build succeeded for 1 out of 1 +== Temporary log file(s) /tmp/eb-oYDvfO/easybuild-zCM5rM.log* have been removed. +== Temporary directory /tmp/eb-oYDvfO has been removed. +``` + +## Test New Module + +```console +$ ml av git/ + +-------------------- /home/kru0052/.local/easybuild/modules/all -------------------- + git/2.19.0 git/2.19.1-Py-3.6-kru0052 (D) + +-------------------- /apps/modules/tools -------------------- + git/2.18.0 + + Where: + D: Default Module + + If you need software that is not listed, request it at support@it4i.cz. + +$ ml git/2.19.1-Py-3.6-foxik +$ git --version +git version 2.19.1 +``` diff --git a/docs.it4i/EasyBuild-Hands-on-2018/template.eb b/docs.it4i/EasyBuild-Hands-on-2018/template.eb new file mode 100644 index 0000000000000000000000000000000000000000..8d0f6ae5078807d5030db88c31ca6e4f7a9a23d4 --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/template.eb @@ -0,0 +1,28 @@ +# IT4Innovations 2018 + +easyblock = + +name = +version = +versionsuffix = + +homepage = +description = """ """ + +toolchain = {'name': '', 'version': ''} + +source_urls = [''] +sources = [''] + +preconfigopts = + +builddependencies = [] + +dependencies = [] + +sanity_check_paths = { + 'files': [], + 'dirs': [], +} + +moduleclass = '' diff --git a/docs.it4i/EasyBuild-Hands-on-2018/troubleshooting.md b/docs.it4i/EasyBuild-Hands-on-2018/troubleshooting.md new file mode 100644 index 0000000000000000000000000000000000000000..00d4e40624d6d0485e03b5717402fc98f883a1dc --- /dev/null +++ b/docs.it4i/EasyBuild-Hands-on-2018/troubleshooting.md @@ -0,0 +1,126 @@ +# Troubleshooting - Typical Problems + +## INSTALL - Already Installed + +Install module from easyconfig `git-2.15.0-GCC-7.1.0-2.28-Python-3.6.1.eb` + +```console +[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb +== temporary log file in case of crash /tmp/eb-KMNHse/easybuild-Jwn_R7.log +== git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK is already installed (module found), skipping +== No easyconfigs left to be built. +== Build succeeded for 0 out of 0 +== Temporary log file(s) /tmp/eb-KMNHse/easybuild-Jwn_R7.log* have been removed. +== Temporary directory /tmp/eb-KMNHse has been removed. +``` + +## REINSTALL - Module Already Loaded + +Load module `git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK` and reinstall module + +```console +[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -f +== temporary log file in case of crash /tmp/eb-NDp_Rx/easybuild-XgYOq9.log + +WARNING: Found one or more non-allowed loaded (EasyBuild-generated) modules in current environment: +* bzip2/1.0.6 +* libreadline/6.3 +* SQLite/3.13.0 +* Tcl/8.6.5 +* Tk/8.6.5 +* GMP/6.1.1 +* XZ/5.2.2 +* zlib/1.2.11 +* Python/3.6.1 +* cURL/7.53.1 +* expat/2.2.0 +* ncurses/6.0 +* gettext/0.19.8.1 +* GCCcore/7.1.0 +* GCCcore/7.1.0 +* binutils/2.28-GCCcore-7.1.0 +* Perl/5.26.0-GCC-7.1.0-2.28-bare +* git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK + +This is not recommended since it may affect the installation procedure(s) performed by EasyBuild. + +To make EasyBuild allow particular loaded modules, use the --allow-loaded-modules configuration option. +To specify action to take when loaded modules are detected, use --detect-loaded-modules={error,ignore,purge,unload,warn}. + +See http://easybuild.readthedocs.io/en/latest/Detecting_loaded_modules.html for more information. + +== processing EasyBuild easyconfig /home_lustre/kru0052/hands-on/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb +== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK... +== fetching files... +== creating build dir, resetting environment... +== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Module is already loaded (EBROOTGIT is set), installation cannot continue. +== Results of the build can be found in the log file(s) /tmp/eb-NDp_Rx/easybuild-git-2.14.1-20170911.085415.EaWTJ.log +ERROR: Build of /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: 'build failed (first 300 chars): Module is already loaded (EBROOTGIT is set), installation cannot continue.') +``` + +## INSTALL/REINSTALL - Irresolvable Dependencies + +Change `('Python', '3.6.1', '', True)` to `('Python', '3.6.1')` and reinstall module + +```console +[kru0052@login2.anselm ~]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -r +== temporary log file in case of crash /tmp/eb-7toWyv/easybuild-koZYcG.log +== processing EasyBuild easyconfig /home_lustre/kru0052/hands-on/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb +== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK... +== fetching files... +== creating build dir, resetting environment... +== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Missing modules for one or more dependencies: Python/3.6.1-GCC-7.1.0-2.28 +== Results of the build can be found in the log file(s) /tmp/eb-7toWyv/easybuild-git-2.14.1-20170911.084653.qAHAA.log +ERROR: Build of /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: 'build failed (first 300 chars): Missing modules for one or more dependencies: Python/3.6.1-GCC-7.1.0-2.28') +``` + +## INSTALL/REINSTALL - Can't Find Path + +Use bad easyconfig name `eb git-2.14.1-GCC-7.1.0-2.28.eb` + +```console +[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28.eb -f +== temporary log file in case of crash /tmp/eb-keyVcY/easybuild-AgLBmT.log +ERROR: Can't find path /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28.eb +``` + +## INSTALL/REINSTALL - Sanity Check Failed + +Change `'files': ['bin/git']` to `'files': ['bin/git2']` + +```console +[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -f +== temporary log file in case of crash /tmp/eb-Lv9b1_/easybuild-h2F2uL.log +== processing EasyBuild easyconfig /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb +== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK... +== fetching files... +== creating build dir, resetting environment... +== unpacking... +== patching... +== preparing... +== configuring... +== building... +== testing... +== installing... +== taking care of extensions... +== postprocessing... +== sanity checking... +== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Sanity check failed: no file of ('bin/git2',) in /home/kru0052/.local/easybuild/software/git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK +== Results of the build can be found in the log file(s) /tmp/eb-Lv9b1_/easybuild-git-2.14.1-20170911.085750.VJyvg.log +ERROR: Build of /home_lustre/kru0052/test-build/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: "build failed (first 300 chars): Sanity check failed: no file of ('bin/git2',) in /home/kru0052/.local/easybuild/software/git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK") +``` + +## INSTALL/REINSTALL - Couldn't Find File + +Change `sources = ['%(name)s-%(version)s.tar.gz']` to `sources = ['%(name)s-%(version)s-test.tar.gz']` + +```console +[kru0052@login2.anselm test-build]$ eb git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb -f +== temporary log file in case of crash /tmp/eb-N64arw/easybuild-lXjC9B.log +== processing EasyBuild easyconfig /home_lustre/kru0052/hands-one/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb +== building and installing git/2.14.1-GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK... +== fetching files... +== FAILED: Installation ended unsuccessfully (build directory: /home/kru0052/.local/easybuild/build/git/2.14.1/GCC-7.1.0-2.28-Python-3.6.1-by-FoXiK): build failed (first 300 chars): Couldn't find file git-2.14.1-test.tar.gz anywhere, and downloading it didn't work either... Paths attempted (in order): /home_lustre/kru0052/hands-on/g/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/hands-on/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/hands-on/git-2.14.1-test.tar.gz, / +== Results of the build can be found in the log file(s) /tmp/eb-N64arw/easybuild-git-2.14.1-20170911.085547.BnfZG.log +ERROR: Build of /home_lustre/kru0052/hands-on/git-2.14.1-GCC-7.1.0-2.28-Python-3.6.1.eb failed (err: "build failed (first 300 chars): Couldn't find file git-2.14.1-test.tar.gz anywhere, and downloading it didn't work either... Paths attempted (in order): /home_lustre/kru0052/hands-on/g/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/test-build/git/git-2.14.1-test.tar.gz, /home_lustre/kru0052/hands-on/git-2.14.1-test.tar.gz, /") +``` diff --git a/mkdocs.yml b/mkdocs.yml index 051cb9e5433c1dc871c73108deb2341caf447e9e..3811f599d7d1c8715c9fd285cfaed51a62374f2c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -15,7 +15,7 @@ repo_name: sccs/docs.it4i.cz repo_url: https://code.it4i.cz/sccs/docs.it4i.cz # Copyright -copyright: Copyright (c) 2013-2018 IT4Innovations__VERSION__ +copyright: Copyright (c) 2013-2018 IT4Innovations / ver. 03ecba66 / 2018-10-16 10:21:23 +0200 pages: - Home: index.md @@ -175,6 +175,14 @@ pages: - ParaView: software/viz/paraview.md - PBS Pro Documentation: pbspro.md - API Documentation: apiv1.md + - Productivity Tools: + - EasyBuild: + - Introduction: EasyBuild-Hands-on-2018/Introduction.md + - Example 1: EasyBuild-Hands-on-2018/firstsoftware.md + - Example 2: EasyBuild-Hands-on-2018/modification.md + - Example 3: EasyBuild-Hands-on-2018/create.md + - Example 4: EasyBuild-Hands-on-2018/easybuildandsingularity.md + - Troubleshooting: EasyBuild-Hands-on-2018/troubleshooting.md extra: repo_icon: gitlab