Hi everyone,
I have been trying for more than a week now to set up a conda virtual environmnet to run my code. Unfortunately, I keep running in errors due to incompatibilities between TensorFlow packages and Keras packages. I have tried recreating new environments with different version of pythons, of keras and of tensorflow but nothing changed (except maybe the kind of error). I even tried the virtual environment from my own laptop (in which everything work) on the cluster, but then it would start throwing errors on hpc05 when importing keras or hyperas.
I have talked with a friend programmer and he thinks the problem might lie in the absence of NVIDIA drivers and problems with the installation of tensorflow-gpu?? Could this be it?
What can I do to get a working environment?
Let me know about any information needed.
Thank you!
1 Like
Please share a minimal environment.yml that reproduces the problem or the list of packages that you are installing. Are you using hpc05 cluster? I don’t remember it having GPUs, or is my information not up to date?
import pandas as pd
import numpy as np
import h5py
import matplotlib.pyplot as plt
import tensorflow as tf
from sklearn.model_selection import train_test_split
from matplotlib import pyplot
import random
from scipy.stats import truncnorm
import json
import csv
from hyperopt import Trials, STATUS_OK, tpe
from hyperas import optim
from hyperas.distributions import choice, uniform
these are the packages I need, and I am having problem with ‘from hyperas import optim’ .
Yes, I am using the hpc05 cluster.
Sorry, I mean the environment, so the specification of the packages that you install with conda.
like, versions and such? Is there a specific command I can run on terminal to have such info? (I am really new to this, sorry!)
I think conda list should work.
1 Like
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
_tflow_select 2.3.0 mkl
absl-py 0.15.0 pyhd3eb1b0_0
aiohttp 3.8.1 py38h7f8727e_1
aiosignal 1.2.0 pyhd3eb1b0_0
argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py38h0a891b7_2 conda-forge
astor 0.8.1 py38h06a4308_0
asttokens 2.0.5 pyhd8ed1ab_0 conda-forge
astunparse 1.6.3 py_0
async-timeout 4.0.1 pyhd3eb1b0_0
attrs 21.4.0 pyhd3eb1b0_0
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.11.1 pyha770c72_0 conda-forge
blas 1.0 mkl
bleach 5.0.1 pyhd8ed1ab_0 conda-forge
blinker 1.4 py38h06a4308_0
brotlipy 0.7.0 py38h27cfd23_1003
c-ares 1.18.1 h7f8727e_0
ca-certificates 2022.6.15 ha878542_0 conda-forge
cachetools 4.2.2 pyhd3eb1b0_0
certifi 2022.6.15 py38h578d9bd_0 conda-forge
cffi 1.15.1 pypi_0 pypi
charset-normalizer 2.0.4 pyhd3eb1b0_0
click 8.0.4 py38h06a4308_0
cloudpickle 2.1.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.5 pyhd8ed1ab_0 conda-forge
cryptography 37.0.1 py38h9ce1e76_0
dataclasses 0.8 pyh6d0b6a4_7
dbus 1.13.18 hb2f20db_0
debugpy 1.6.2 pypi_0 pypi
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
executing 0.8.3 pyhd8ed1ab_0 conda-forge
expat 2.4.8 h27087fc_0 conda-forge
fastjsonschema 2.15.3 pypi_0 pypi
flit-core 3.7.1 pyhd8ed1ab_0 conda-forge
fontconfig 2.14.0 h8e229c2_0 conda-forge
freetype 2.10.4 h0708190_1 conda-forge
frozenlist 1.2.0 py38h7f8727e_0
future 0.18.2 py38h578d9bd_5 conda-forge
gast 0.3.3 pypi_0 pypi
gettext 0.19.8.1 hf34092f_1004 conda-forge
glib 2.58.3 py38h73cb85d_1004 conda-forge
google-auth 2.6.0 pyhd3eb1b0_0
google-auth-oauthlib 0.4.4 pyhd3eb1b0_0
google-pasta 0.2.0 pyhd3eb1b0_0
grpcio 1.32.0 pypi_0 pypi
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
h5py 2.10.0 py38hd6299e0_1
hdf5 1.10.6 hb1b8bf9_0
hyperas 0.4.1 pyh8c360ce_1 conda-forge
hyperopt 0.2.7 pyhd8ed1ab_0 conda-forge
icu 58.2 hf484d3e_1000 conda-forge
idna 3.3 pyhd3eb1b0_0
importlib-metadata 4.11.3 py38h06a4308_0
importlib_resources 5.8.0 pyhd8ed1ab_0 conda-forge
intel-openmp 2021.4.0 h06a4308_3561
ipykernel 6.15.1 pyh210e3f2_0 conda-forge
ipython 8.4.0 py38h578d9bd_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
ipywidgets 7.7.1 pyhd8ed1ab_0 conda-forge
jedi 0.18.1 py38h578d9bd_1 conda-forge
jinja2 3.0.3 pyhd3eb1b0_0
jpeg 9e h166bdaf_1 conda-forge
jsonschema 4.7.2 pyhd8ed1ab_0 conda-forge
jupyter 1.0.0 py38h578d9bd_7 conda-forge
jupyter-client 7.3.4 pypi_0 pypi
jupyter_client 7.0.6 pyhd8ed1ab_0 conda-forge
jupyter_console 6.4.4 pyhd8ed1ab_0 conda-forge
jupyter_core 4.11.1 py38h578d9bd_0 conda-forge
jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge
jupyterlab_widgets 1.1.1 pyhd8ed1ab_0 conda-forge
keras 2.3.1 0
keras-applications 1.0.8 py_1
keras-base 2.3.1 py38_0
keras-preprocessing 1.1.2 pyhd3eb1b0_0
ld_impl_linux-64 2.38 h1181459_1
libedit 3.1.20210910 h7f8727e_0
libffi 3.2.1 hf484d3e_1007
libgcc-ng 11.2.0 h1234567_1
libgfortran-ng 7.5.0 ha8ba4b0_17
libgfortran4 7.5.0 ha8ba4b0_17
libgomp 11.2.0 h1234567_1
libiconv 1.16 h516909a_0 conda-forge
libpng 1.6.37 h21135ba_2 conda-forge
libprotobuf 3.20.1 h4ff587b_0
libsodium 1.0.18 h36c2ea0_1 conda-forge
libstdcxx-ng 11.2.0 h1234567_1
libuuid 2.32.1 h7f98852_1000 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxml2 2.9.14 h74e7548_0
markdown 3.3.4 py38h06a4308_0
markupsafe 2.1.1 py38h7f8727e_0
matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge
mistune 0.8.4 py38h497a2fe_1005 conda-forge
mkl 2021.4.0 h06a4308_640
mkl-service 2.4.0 py38h7f8727e_0
mkl_fft 1.3.1 py38hd3c417c_0
mkl_random 1.2.2 py38h51133e4_0
multidict 5.2.0 py38h7f8727e_2
nbclient 0.6.6 pyhd8ed1ab_0 conda-forge
nbconvert 6.5.0 pyhd8ed1ab_0 conda-forge
nbconvert-core 6.5.0 pyhd8ed1ab_0 conda-forge
nbconvert-pandoc 6.5.0 pyhd8ed1ab_0 conda-forge
nbformat 5.4.0 pyhd8ed1ab_0 conda-forge
ncurses 6.3 h5eee18b_3
nest-asyncio 1.5.5 pyhd8ed1ab_0 conda-forge
networkx 2.8.5 pyhd8ed1ab_0 conda-forge
notebook 6.4.12 pyha770c72_0 conda-forge
numpy 1.19.5 pypi_0 pypi
numpy-base 1.22.3 py38hf524024_0
oauthlib 3.1.0 py_0
openssl 1.1.1l h7f98852_0 conda-forge
opt_einsum 3.3.0 pyhd3eb1b0_1
packaging 21.3 pyhd8ed1ab_0 conda-forge
pandoc 2.18 ha770c72_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pcre 8.45 h9c3ff4c_0 conda-forge
pexpect 4.8.0 pyh9f0ad1d_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pip 22.1.2 py38h06a4308_0
prometheus_client 0.14.1 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.30 pyha770c72_0 conda-forge
prompt_toolkit 3.0.30 hd8ed1ab_0 conda-forge
protobuf 3.20.1 py38h295c915_0
psutil 5.9.1 py38h0a891b7_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
py4j 0.10.9.5 pyhd8ed1ab_0 conda-forge
pyasn1 0.4.8 pyhd3eb1b0_0
pyasn1-modules 0.2.8 py_0
pycparser 2.21 pyhd3eb1b0_0
pygments 2.12.0 pyhd8ed1ab_0 conda-forge
pyjwt 2.4.0 py38h06a4308_0
pyopenssl 22.0.0 pyhd3eb1b0_0
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pyqt 5.9.2 py38h05f1152_4
pyrsistent 0.18.1 py38h0a891b7_1 conda-forge
pysocks 1.7.1 py38h06a4308_0
python 3.8.1 h0371630_1
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.16.1 pyhd8ed1ab_0 conda-forge
python-flatbuffers 2.0 pyhd3eb1b0_0
python_abi 3.8 2_cp38 conda-forge
pyyaml 6.0 py38h7f8727e_1
pyzmq 19.0.2 py38ha71036d_2 conda-forge
qt 5.9.7 h5867ecd_1
qtconsole 5.3.1 pyhd8ed1ab_0 conda-forge
qtconsole-base 5.3.1 pyha770c72_0 conda-forge
qtpy 2.1.0 pyhd8ed1ab_0 conda-forge
readline 7.0 h7b6447c_5
requests 2.28.1 py38h06a4308_0
requests-oauthlib 1.3.0 py_0
rsa 4.7.2 pyhd3eb1b0_1
scipy 1.7.3 py38hc147768_0
send2trash 1.8.0 pyhd8ed1ab_0 conda-forge
setuptools 61.2.0 py38h06a4308_0
sip 4.19.13 py38h295c915_0
six 1.15.0 pypi_0 pypi
soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge
sqlite 3.33.0 h62c20be_0
stack_data 0.3.0 pyhd8ed1ab_0 conda-forge
tensorboard 2.4.0 pyhc547734_0
tensorboard-plugin-wit 1.6.0 py_0
tensorflow 2.4.1 mkl_py38hb2083e0_0
tensorflow-base 2.4.1 mkl_py38h43e0292_0
tensorflow-estimator 2.4.0 pypi_0 pypi
termcolor 1.1.0 py38h06a4308_1
terminado 0.15.0 py38h578d9bd_0 conda-forge
tinycss2 1.1.1 pyhd8ed1ab_0 conda-forge
tk 8.6.12 h1ccaba5_0
tornado 6.2 pypi_0 pypi
tqdm 4.64.0 pyhd8ed1ab_0 conda-forge
traitlets 5.3.0 pyhd8ed1ab_0 conda-forge
typing-extensions 3.7.4.3 pypi_0 pypi
typing_extensions 4.1.1 pyh06a4308_0
urllib3 1.26.9 py38h06a4308_0
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 pypi_0 pypi
werkzeug 2.0.3 pyhd3eb1b0_0
wheel 0.37.1 pyhd3eb1b0_0
widgetsnbextension 3.6.1 pyha770c72_0 conda-forge
wrapt 1.12.1 pypi_0 pypi
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xz 5.2.5 h7f8727e_1
yaml 0.2.5 h7b6447c_0
yarl 1.6.3 py38h27cfd23_0
zeromq 4.3.4 h9c3ff4c_1 conda-forge
zipp 3.8.0 py38h06a4308_0
zlib 1.2.12 h7f8727e_2
here there are
Can you try with python 3.10? You are using 3.8, which is very old by now, and packages might get obsolete.
I had tried with python 3.9 without success, now I am gonna try with python 3.10 as you suggested