Comp:cluster reservation - Revision history

Gerritg: /* Tutorial and Quick Start (for bash users) */

2021-02-19T21:30:19Z

Tutorial and Quick Start (for bash users)

← Older revision		Revision as of 21:30, 19 February 2021
Line 55:		Line 55:
	are available on <tt>cn31</tt>, use:		are available on <tt>cn31</tt>, use:

	sbatch -n 40 --mem=521GB -w cn31 ~~myjob~~.sh		sbatch -n 40 --mem=521GB -w cn31 my_job.sh

	Your job <tt>myjob.sh</tt> should update the table of reservations when it starts, so it should		Your job <tt>myjob.sh</tt> should update the table of reservations when it starts, so it should
Line 65:		Line 65:
	you made a reservation for the node on which your jobs is running.		you made a reservation for the node on which your jobs is running.

	Your reservation expires the number of days you reserved ~~after~~ your job ~~started~~. ~~You can also~~		<b>Note</b> the job that uses the reserved node should NOT be submitted with the <tt>--exclude<tt> flag. Instead, make sure it runs on the reserved node using the <tt>-w</tt> flag:
	cancel ~~the~~ reservation ~~with~~
			Cn reserve cn50 7
			sbatch -w cn50 -n 24 my_job.sh

			Your reservation expires the number of days you reserved AFTER your job STARTED.

			To cancel a reservation use:

	Cn cancel cn31		Cn cancel cn31

Gerritg: /* Tutorial and Quick Start (for bash users) */

2021-02-19T21:20:32Z

Tutorial and Quick Start (for bash users)

← Older revision		Revision as of 21:20, 19 February 2021
Line 72:		Line 72:
	You can add extra days to the reservation with:		You can add extra days to the reservation with:

	Dn add cn31 3		Cn add cn31 3

	You can also use a negative number of days to shorten the reservation.		You can also use a negative number of days to shorten the reservation.

Gerritg: /* Tutorial and Quick Start (for bash users) */ Cn add command

2021-02-19T21:20:12Z

Tutorial and Quick Start (for bash users): Cn add command

← Older revision		Revision as of 21:20, 19 February 2021
Line 70:		Line 70:
	Cn cancel cn31		Cn cancel cn31

	You can add or update the comment of your reservation ~~with~~:		You can add extra days to the reservation with:

			Dn add cn31 3

			You can also use a negative number of days to shorten the reservation.

			To add or update the comment of your reservation use:

	Cn comment cn31 "Thanks!"		Cn comment cn31 "Thanks!"

Dewijs: tchem --> thchem

2021-02-17T16:16:24Z

tchem --> thchem

← Older revision		Revision as of 16:16, 17 February 2021
Line 7:		Line 7:
	Make sure to have <tt>/vol/thchem/bin</tt> in your bash PATH variable. In your <tt>.bashrc</tt> add		Make sure to have <tt>/vol/thchem/bin</tt> in your bash PATH variable. In your <tt>.bashrc</tt> add

	export PATH=$PATH:/vol/~~tchem~~/bin		export PATH=$PATH:/vol/thchem/bin

	On one of the cluster nodes and on <tt>lilo7</tt> (but not <tt>lilo6</tt> and older) check this:		On one of the cluster nodes and on <tt>lilo7</tt> (but not <tt>lilo6</tt> and older) check this:

Gerritg: add .pfofile example

2021-02-17T09:12:58Z

add .pfofile example

← Older revision		Revision as of 09:12, 17 February 2021
Line 17:		Line 17:
	You are connected to database "batchq" as user "cnuser" on host "cn58" (address "131.174.30.158") at port "16034".		You are connected to database "batchq" as user "cnuser" on host "cn58" (address "131.174.30.158") at port "16034".

	If you get <tt>Cn: command not found</tt>, ~~ask someone to setup~~ a <tt>.profile</tt> ~~script~~.		If you get <tt>Cn: command not found</tt>, create a file with the name <tt>.profile</tt> in your home directory. The file should contain these four lines:

			. /system.profile
			if [ "$SHELL" == /bin/bash ]; then
			test -s ~/.bashrc && . ~/.bashrc \|\| true
			fi

	From the command line you can get the manual by just typing <tt>Cn</tt>, or		From the command line you can get the manual by just typing <tt>Cn</tt>, or

Gerritg at 21:22, 10 February 2021

2021-02-10T21:22:58Z

← Older revision		Revision as of 21:22, 10 February 2021
Line 9:		Line 9:
	export PATH=$PATH:/vol/tchem/bin		export PATH=$PATH:/vol/tchem/bin

	On one of the ~~clusternodes~~ and on <tt>lilo7</tt> (but not <tt>lilo6</tt> and older) check this:		On one of the cluster nodes and on <tt>lilo7</tt> (but not <tt>lilo6</tt> and older) check this:

	Cn info		Cn info

Gerritg: fix link

2021-02-10T21:20:44Z

fix link

← Older revision		Revision as of 21:20, 10 February 2021
Line 1:		Line 1:
	Batch jobs are handled by [https://wiki.science.ru.nl/slurm SLURM]. However, sometimes you need all cores of one specific cluster node and you want to ask other theoretical chemistry group members not to start new jobs on that node. We first used [https://wiki.theochem.ru.nl Slack/Yoink] to request nodes, but we now have a script to automate this process. The script will put the cluster node reservation requests in a database and from a bash script you can get a list of cluster nodes that have been reserved.		Batch jobs are handled by [https://wiki.science.ru.nl/slurm SLURM]. However, sometimes you need all cores of one specific cluster node and you want to ask other theoretical chemistry group members not to start new jobs on that node. We first used [https://wiki.theochem.ru.nl/Comp:clusters Slack/Yoink] to request nodes, but we now have a script to automate this process. The script will put the cluster node reservation requests in a database and from a bash script you can get a list of cluster nodes that have been reserved.

	There can be only one reservation at at time for each cluster node.		There can be only one reservation at at time for each cluster node.

Gerritg: link to old Slack/Yoink

2021-02-10T21:20:02Z

link to old Slack/Yoink

← Older revision		Revision as of 21:20, 10 February 2021
Line 1:		Line 1:
	Batch jobs are handled by [https://wiki.science.ru.nl/slurm SLURM]. However, sometimes you need all cores of one specific cluster node and you want to ask other theoretical chemistry group members not to start new jobs on that node. We first used Slack/Yoink to request nodes, but we now have a script to automate this process. The script will put the cluster node reservation requests in a database and from a bash script you can get a list of cluster nodes that have been reserved.		Batch jobs are handled by [https://wiki.science.ru.nl/slurm SLURM]. However, sometimes you need all cores of one specific cluster node and you want to ask other theoretical chemistry group members not to start new jobs on that node. We first used [https://wiki.theochem.ru.nl Slack/Yoink] to request nodes, but we now have a script to automate this process. The script will put the cluster node reservation requests in a database and from a bash script you can get a list of cluster nodes that have been reserved.

	There can be only one reservation at at time for each cluster node.		There can be only one reservation at at time for each cluster node.

Gerritg: /* What happens when the database is down? */

2021-02-10T21:14:06Z

What happens when the database is down?

← Older revision		Revision as of 21:14, 10 February 2021
Line 118:		Line 118:
	still show the filenames where the commands were recorded, but they will be labeled as <tt>stale</tt>,		still show the filenames where the commands were recorded, but they will be labeled as <tt>stale</tt>,
	since it is not certain when the database stopped working.		since it is not certain when the database stopped working.

			Commands given while the system is down will also automatically generate an e-mail to <tt>gerritg@theochem.ru.nl</tt>.

Gerritg at 21:05, 10 February 2021

2021-02-10T21:05:35Z

← Older revision		Revision as of 21:05, 10 February 2021
Line 82:		Line 82:

	sbatch --exclude "$EXCL,cn58,cn86" --mem=32GB myjob.sh		sbatch --exclude "$EXCL,cn58,cn86" --mem=32GB myjob.sh

			=== What happens when the database is down? ===
			To check whether the system is working:

			Cn status

			If things are fine you will get:

			/vol/thchem/bin/Cn: The cluster node reservation system is up and running.

			However, when the database is down, some things may still work. Unless things went horribly wrong,
			a comma-separated file with the reservations was written to disk, and also the list of excluded
			files is on disk.

			In particular, the <tt>Cn exclude</tt> command will give this last known list of excluded nodes. It will
			also give an error message on standard-error (<tt>/dev/stderr</tt>), but that will not interfere
			with this line in your job, since it only uses the cluster node list that is written to standard-output
			<tt>/dev/stdout</tt>:

			EXCL=$(Cn exclude)

			The <tt>Cn show</tt> command will show the last known table with reservations.

			The command

			Cn start

			however, will not be able to update the table. Your commands, however, will be written to a file, and
			they can later be entered "by hand" by whoever fixed the database problem. The list of recorded
			command can be printed:

			Cn recorded

			This will only work well when the system was properly shut down. In case of an unintended crash, it will
			still show the filenames where the commands were recorded, but they will be labeled as <tt>stale</tt>,
			since it is not certain when the database stopped working.