# Paths for loading functions from a python "library"

**URL:** https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58
**Category:** Programming questions
**Created:** [15 May 2020 07:44 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58 "2020-05-15T07:44:14Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![gsteele13](https://forum.kavli.tudelft.nl/letter_avatar_proxy/v4/letter/g/3d9bf3/32.png) [@gsteele13](https://forum.kavli.tudelft.nl/u/gsteele13)
#### Post date: [15 May 2020 07:44 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/1 "2020-05-15T07:44:15Z")

</div>

Question: I’ve been working on the “datacube” library.

> **[Files · master · SteeleLab / datacube · GitLab](https://gitlab.tudelft.nl/steelelab/datacube/tree/master)**
>
> Delft University of Technology GitLab server

I’ve never made a “library” before in python, so I’m a bit of a noob, and I’m a bit confused about paths.

In my library, I keep the code for the class in a file `datacube.py`

If I am working inside the repo directory, I can do the following: `from datacube import Datacube`

I have now also linked the repo directory to `~/lib` on the JH (which is by default in the path for all users BTW)

However, working from outside the repo folder, I need to run `from datacube.datacube import Datacube`

Is there some way to be able to load the library using `from datacube import Datacube` when I am working outside the repo?

Thanks!

---

<div class="post-metadata">

### Author: ![anton-akhmerov](https://forum.kavli.tudelft.nl/user_avatar/forum.kavli.tudelft.nl/anton-akhmerov/32/4_2.png) [@anton-akhmerov](https://forum.kavli.tudelft.nl/u/anton-akhmerov)
#### Post date: [15 May 2020 08:59 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/2 "2020-05-15T08:59:12Z")

</div>

Nice library!

There may be some trick to deal with the paths, but I think ultimately even in the short term, I think your best solution is to make your library pip-installable by adding a minimal `setup.py`. It sounds like a lot of extra work, but actually isn’t.

Here’s all you need to provide ([source](https://python-packaging.readthedocs.io/en/latest/minimal.html)):

```python
from setuptools import setup

setup(name='funniest',
      version='0.1',
      description='The funniest joke in the world',
      url='http://github.com/storborg/funniest',
      author='Flying Circus',
      author_email='flyingcircus@example.com',
      license='MIT',
      packages=['funniest'],
      zip_safe=False)

```

You can still create symlink the project folder with the installed version by using `pip install . --editable`.

---

<div class="post-metadata">

### Author: ![gsteele13](https://forum.kavli.tudelft.nl/letter_avatar_proxy/v4/letter/g/3d9bf3/32.png) [@gsteele13](https://forum.kavli.tudelft.nl/u/gsteele13)
#### Post date: [15 May 2020 12:00 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/3 "2020-05-15T12:00:10Z")

</div>

But let’s say that I want to use the gitlab “bleeding edge” code in my notebooks that I pull to ~/lib.

Is there a solution for that?

---

<div class="post-metadata">

### Author: ![anton-akhmerov](https://forum.kavli.tudelft.nl/user_avatar/forum.kavli.tudelft.nl/anton-akhmerov/32/4_2.png) [@anton-akhmerov](https://forum.kavli.tudelft.nl/u/anton-akhmerov)
#### Post date: [15 May 2020 12:34 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/4 "2020-05-15T12:34:18Z")

</div>

Even then I recommend to install code not just in an arbitrary folder, but somewhere, where pip recognizes this, e.g. a shared conda environment.

This you can easily achieve installing bleeding edge code with `pip install git+https://gitlab.tudelft.nl/steelelab/datacube.git@master` (the part after `@` can be any reference that git recognizes.

---

<div class="post-metadata">

### Author: ![gsteele13](https://forum.kavli.tudelft.nl/letter_avatar_proxy/v4/letter/g/3d9bf3/32.png) [@gsteele13](https://forum.kavli.tudelft.nl/u/gsteele13)
#### Post date: [15 May 2020 13:05 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/5 "2020-05-15T13:05:06Z")

</div>

you’re strict! i’ll have a look at it

---

<div class="post-metadata">

### Author: ![anton-akhmerov](https://forum.kavli.tudelft.nl/user_avatar/forum.kavli.tudelft.nl/anton-akhmerov/32/4_2.png) [@anton-akhmerov](https://forum.kavli.tudelft.nl/u/anton-akhmerov)
#### Post date: [15 May 2020 13:05 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/6 "2020-05-15T13:05:59Z")

</div>

It’s not really about strictness; rather before long this actually saves you work through avoiding headache.

---

<div class="post-metadata">

### Author: ![gsteele13](https://forum.kavli.tudelft.nl/letter_avatar_proxy/v4/letter/g/3d9bf3/32.png) [@gsteele13](https://forum.kavli.tudelft.nl/u/gsteele13)
#### Post date: [19 May 2020 14:05 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/7 "2020-05-19T14:05:17Z")

</div>

Thanks, it was indeed not that hard 🙂

I guess though that for it to be PIP installable for anybody via PyPI (without downloading the source), I just have to make it a public repo and then register it.

Are there general guidelines for putting stuff on PyPI?

(This is a pretty silly and specific library, but it might be still handy to be able to just pip install it)

Thanks,  
Gary

---

<div class="post-metadata">

### Author: ![anton-akhmerov](https://forum.kavli.tudelft.nl/user_avatar/forum.kavli.tudelft.nl/anton-akhmerov/32/4_2.png) [@anton-akhmerov](https://forum.kavli.tudelft.nl/u/anton-akhmerov)
#### Post date: [19 May 2020 15:06 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/8 "2020-05-19T15:06:55Z")

</div>

See the pip command I shared above: to make a repository installable without cloning you don’t need to publish it on pypi: pip understands how to deal with git sources directly.

> [@gsteele13](#):
>
> Are there general guidelines for putting stuff on PyPI?

Definitely. You’ll need to provide some extra information in the setup, and you’ll need to upload the resulting package to pypi using `twine`.

To upload a new version to pypi you’ll need to run these commands:

```bash
python setup.py sdist
python setup.py bdist_wheel
# check that the tarballs in ./dist have everything needed and that the wheel is universal
twine upload dist/*

```

But then you’ll also want to manage versioning of your package, to make that simple, I suggest to get started with [miniver](https://github.com/jbweston/miniver) (written by Joe).

---

<div class="post-metadata">

### Author: ![gsteele13](https://forum.kavli.tudelft.nl/letter_avatar_proxy/v4/letter/g/3d9bf3/32.png) [@gsteele13](https://forum.kavli.tudelft.nl/u/gsteele13)
#### Post date: [29 May 2020 07:05 UTC](https://forum.kavli.tudelft.nl/t/paths-for-loading-functions-from-a-python-library/58/9 "2020-05-29T07:05:58Z")

</div>

OK, thanks, for now I’ll stick to pip installs from the git dir 🙂
