# Parallelising a qutip for loop over the cluster

**URL:** https://forum.kavli.tudelft.nl/t/parallelising-a-qutip-for-loop-over-the-cluster/364
**Category:** Cluster
**Created:** [22 June 2023 14:23 UTC](https://forum.kavli.tudelft.nl/t/parallelising-a-qutip-for-loop-over-the-cluster/364 "2023-06-22T14:23:10Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![gsteele13](https://forum.kavli.tudelft.nl/letter_avatar_proxy/v4/letter/g/3d9bf3/32.png) [@gsteele13](https://forum.kavli.tudelft.nl/u/gsteele13)
#### Post date: [22 June 2023 14:23 UTC](https://forum.kavli.tudelft.nl/t/parallelising-a-qutip-for-loop-over-the-cluster/364/1 "2023-06-22T14:23:10Z")

</div>

Hi all,

We have been running some code that basically runs [QuTiP](https://qutip.org/)’s `steadystate()` for different input parameters. Because of a need for an increased Hilbert space size, each of these is now taking 4 hours on our laptop computers, and so we would like to explore running them on the cluster.

Some googling suggests that the qutip’s `parfor` is a basic staring point:

[https://qutip.org/docs/latest/guide/guide-parfor.html](https://qutip.org/docs/latest/guide/guide-parfor.html)

But reading further, it seems that qutip also has an implementation that can use ipython parallelisation across multiple hosts:

[Functions — QuTiP 4.7 Documentation](https://qutip.org/docs/latest/apidoc/functions.html#qutip.ipynbtools.parallel_map)

And now my question:

- Will `qutip.parallel_map()` directly be able to distribute compute tasks across the nodes of the cluster?

If yes, then my follow-up question is: how do I do this in practice? Do I log into the `hpc05` node and batch a job based on the execution of a python file that calls `qutip.parallel_map()` and then magic just happens?

Thanks!  
Gary

---

<div class="post-metadata">

### Author: ![anton-akhmerov](https://forum.kavli.tudelft.nl/user_avatar/forum.kavli.tudelft.nl/anton-akhmerov/32/4_2.png) [@anton-akhmerov](https://forum.kavli.tudelft.nl/u/anton-akhmerov)
#### Post date: [23 June 2023 14:14 UTC](https://forum.kavli.tudelft.nl/t/parallelising-a-qutip-for-loop-over-the-cluster/364/2 "2023-06-23T14:14:56Z")

</div>

`parfor` is definitely not what you want, because it says:

> Executes a multi-variable function in parallel on the local machine.

To use a parallel\_map, you would need to launch an `ipyparallel` cluster through pbs. My group uses [distributed](https://distributed.dask.org/en/stable/) to launch parallel computations on a cluster from the group jupyterhub. For that we use a small wrapper [Quantum Tinkerer / dask-quantumtinkerer · GitLab](https://gitlab.kwant-project.org/qt/dask-quantumtinkerer) that is specific to our environment.

---

<div class="post-metadata">

### Author: ![gsteele13](https://forum.kavli.tudelft.nl/letter_avatar_proxy/v4/letter/g/3d9bf3/32.png) [@gsteele13](https://forum.kavli.tudelft.nl/u/gsteele13)
#### Post date: [28 June 2023 13:46 UTC](https://forum.kavli.tudelft.nl/t/parallelising-a-qutip-for-loop-over-the-cluster/364/3 "2023-06-28T13:46:00Z")

</div>

OK, thanks, sounds complicated. I’ll look more at what Marios did, I think that is more at our IT deployment expertise level.