Requirements conflicts


#1

Hi,
I’m observing issue with installation of neptune-cli with other packages, sample command line which causes issue pasted below. It overwrites requests with older version which is incompatible with google-cloud-storage package. Just wondering - why neptune-cli requires exact package versions not lower bound?

Also git-review have some strange requirements.

My environment: Ubuntu 16.04, python 2.7.12

Error lines:

git-review 1.25.0 requires argparse, which is not installed.
google-api-core 1.1.0 has requirement requests<3.0.0dev,>=2.18.0, but you'll have requests 2.11.1 which is incompatible.

Full log:

➜  ~ mkvirtualenv sandbox -p `which python2`
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/pawel/.virtualenvs/sandbox/bin/python2
Not overwriting existing python script /home/pawel/.virtualenvs/sandbox/bin/python (you must use /home/pawel/.virtualenvs/sandbox/bin/python2)
Installing setuptools, pkg_resources, pip, wheel...done.
➜  ~ which python
/home/pawel/.virtualenvs/sandbox/bin/python
➜  ~ pip install google-cloud-storage
Collecting google-cloud-storage
  Using cached google_cloud_storage-1.8.0-py2.py3-none-any.whl
Collecting google-resumable-media>=0.3.1 (from google-cloud-storage)
  Using cached google_resumable_media-0.3.1-py2.py3-none-any.whl
Collecting google-cloud-core<0.29dev,>=0.28.0 (from google-cloud-storage)
  Using cached google_cloud_core-0.28.1-py2.py3-none-any.whl
Collecting google-api-core<2.0.0dev,>=0.1.1 (from google-cloud-storage)
  Using cached google_api_core-1.1.0-py2.py3-none-any.whl
Collecting six (from google-resumable-media>=0.3.1->google-cloud-storage)
  Using cached six-1.11.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=34.0.0 in ./.virtualenvs/sandbox/lib/python2.7/site-packages (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage) (39.0.1)
Collecting protobuf>=3.0.0 (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0.0dev,>=0.4.0 (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached google_auth-1.4.1-py2.py3-none-any.whl
Collecting futures>=3.2.0; python_version < "3.2" (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached futures-3.2.0-py2-none-any.whl
Collecting pytz (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached pytz-2018.3-py2.py3-none-any.whl
Collecting googleapis-common-protos<2.0dev,>=1.5.3 (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
Collecting requests<3.0.0dev,>=2.18.0 (from google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached cachetools-2.0.1-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached certifi-2018.1.18-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached idna-2.6-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached urllib3-1.22-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa>=3.1.4->google-auth<2.0.0dev,>=0.4.0->google-api-core<2.0.0dev,>=0.1.1->google-cloud-storage)
  Using cached pyasn1-0.4.2-py2.py3-none-any.whl
Installing collected packages: six, google-resumable-media, protobuf, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, futures, pytz, googleapis-common-protos, certifi, chardet, idna, urllib3, requests, google-api-core, google-cloud-core, google-cloud-storage
Successfully installed cachetools-2.0.1 certifi-2018.1.18 chardet-3.0.4 futures-3.2.0 google-api-core-1.1.0 google-auth-1.4.1 google-cloud-core-0.28.1 google-cloud-storage-1.8.0 google-resumable-media-0.3.1 googleapis-common-protos-1.5.3 idna-2.6 protobuf-3.5.2.post1 pyasn1-0.4.2 pyasn1-modules-0.2.1 pytz-2018.3 requests-2.18.4 rsa-3.4.2 six-1.11.0 urllib3-1.22
➜  ~ pip install neptune-cli
Collecting neptune-cli
Collecting future==0.16.0 (from neptune-cli)
Collecting more-itertools==4.1.0 (from neptune-cli)
  Using cached more_itertools-4.1.0-py2-none-any.whl
Collecting voluptuous==0.9.3 (from neptune-cli)
Collecting PyYAML==3.12 (from neptune-cli)
Requirement already satisfied: setuptools>=20.3.1 in ./.virtualenvs/sandbox/lib/python2.7/site-packages (from neptune-cli) (39.0.1)
Collecting requests-oauthlib==0.8.0 (from neptune-cli)
  Using cached requests_oauthlib-0.8.0-py2.py3-none-any.whl
Collecting certifi==2016.2.28 (from neptune-cli)
  Using cached certifi-2016.2.28-py2.py3-none-any.whl
Collecting tqdm==4.11.2 (from neptune-cli)
  Using cached tqdm-4.11.2-py2.py3-none-any.whl
Collecting kitchen==1.2.4 (from neptune-cli)
Collecting psutil==5.4.3 (from neptune-cli)
Collecting PyJWT==1.5.2 (from neptune-cli)
  Using cached PyJWT-1.5.2-py2.py3-none-any.whl
Collecting raven==6.1.0 (from neptune-cli)
  Using cached raven-6.1.0-py2.py3-none-any.whl
Collecting humanize==0.5.1 (from neptune-cli)
Collecting six==1.10.0 (from neptune-cli)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting pathlib2==2.3.0 (from neptune-cli)
  Using cached pathlib2-2.3.0-py2.py3-none-any.whl
Collecting mock==2.0.0 (from neptune-cli)
  Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting terminaltables==2.1.0 (from neptune-cli)
Collecting gitdb==0.6.4 (from neptune-cli)
Collecting websocket-client==0.35.0 (from neptune-cli)
Collecting pykwalify==1.5.2 (from neptune-cli)
Collecting Pillow>=1.7.6 (from neptune-cli)
  Using cached Pillow-5.1.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting GitPython==2.0.8 (from neptune-cli)
Collecting tornado==4.2 (from neptune-cli)
Collecting urllib3==1.16 (from neptune-cli)
  Using cached urllib3-1.16-py2.py3-none-any.whl
Collecting Flask==0.12.0 (from neptune-cli)
  Using cached Flask-0.12-py2.py3-none-any.whl
Collecting python-dateutil==2.4.2 (from neptune-cli)
  Using cached python_dateutil-2.4.2-py2.py3-none-any.whl
Collecting git-review==1.25.0 (from neptune-cli)
  Using cached git_review-1.25.0-py2.py3-none-any.whl
Collecting requests==2.11.1 (from neptune-cli)
  Using cached requests-2.11.1-py2.py3-none-any.whl
Collecting enum34==1.1.6 (from neptune-cli)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting oauthlib>=0.6.2 (from requests-oauthlib==0.8.0->neptune-cli)
  Using cached oauthlib-2.0.7-py2.py3-none-any.whl
Collecting contextlib2 (from raven==6.1.0->neptune-cli)
  Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Collecting scandir; python_version < "3.5" (from pathlib2==2.3.0->neptune-cli)
Collecting pbr>=0.11 (from mock==2.0.0->neptune-cli)
  Using cached pbr-4.0.1-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock==2.0.0->neptune-cli)
  Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting smmap>=0.8.5 (from gitdb==0.6.4->neptune-cli)
Collecting docopt>=0.6.2 (from pykwalify==1.5.2->neptune-cli)
Collecting backports.ssl-match-hostname (from tornado==4.2->neptune-cli)
Collecting Jinja2>=2.4 (from Flask==0.12.0->neptune-cli)
  Using cached Jinja2-2.10-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from Flask==0.12.0->neptune-cli)
  Using cached Werkzeug-0.14.1-py2.py3-none-any.whl
Collecting click>=2.0 (from Flask==0.12.0->neptune-cli)
  Using cached click-6.7-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from Flask==0.12.0->neptune-cli)
Requirement already satisfied: argparse in /usr/lib/python2.7 (from git-review==1.25.0->neptune-cli) (1.2.1)
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask==0.12.0->neptune-cli)
tornado 4.2 requires backports.ssl-match-hostname, which is not installed.
git-review 1.25.0 requires argparse, which is not installed.
google-api-core 1.1.0 has requirement requests<3.0.0dev,>=2.18.0, but you'll have requests 2.11.1 which is incompatible.
Installing collected packages: future, six, more-itertools, voluptuous, PyYAML, oauthlib, requests, requests-oauthlib, certifi, tqdm, kitchen, psutil, PyJWT, contextlib2, raven, humanize, scandir, pathlib2, pbr, funcsigs, mock, terminaltables, smmap, gitdb, websocket-client, docopt, python-dateutil, pykwalify, Pillow, GitPython, backports.ssl-match-hostname, tornado, urllib3, MarkupSafe, Jinja2, Werkzeug, click, itsdangerous, Flask, git-review, enum34, neptune-cli
  Found existing installation: six 1.11.0
    Uninstalling six-1.11.0:
      Successfully uninstalled six-1.11.0
  Found existing installation: requests 2.18.4
    Uninstalling requests-2.18.4:
      Successfully uninstalled requests-2.18.4
  Found existing installation: certifi 2018.1.18
    Uninstalling certifi-2018.1.18:
      Successfully uninstalled certifi-2018.1.18
  Found existing installation: urllib3 1.22
    Uninstalling urllib3-1.22:
      Successfully uninstalled urllib3-1.22
Successfully installed Flask-0.12 GitPython-2.0.8 Jinja2-2.10 MarkupSafe-1.0 Pillow-5.1.0 PyJWT-1.5.2 PyYAML-3.12 Werkzeug-0.14.1 backports.ssl-match-hostname-3.5.0.1 certifi-2016.2.28 click-6.7 contextlib2-0.5.5 docopt-0.6.2 enum34-1.1.6 funcsigs-1.0.2 future-0.16.0 git-review-1.25.0 gitdb-0.6.4 humanize-0.5.1 itsdangerous-0.24 kitchen-1.2.4 mock-2.0.0 more-itertools-4.1.0 neptune-cli-2.7.7 oauthlib-2.0.7 pathlib2-2.3.0 pbr-4.0.1 psutil-5.4.3 pykwalify-1.5.2 python-dateutil-2.4.2 raven-6.1.0 requests-2.11.1 requests-oauthlib-0.8.0 scandir-1.7 six-1.10.0 smmap-0.9.0 terminaltables-2.1.0 tornado-4.2 tqdm-4.11.2 urllib3-1.16 voluptuous-0.9.3 websocket-client-0.35.0

#2

It is even bigger problem when forced to use old neptune-cli package (ex. v1.6.3) or when neptune-cli is in requirements.txt: ex.

neptune-cli
PyYAML
fabric3
path.py
jinja2
six
addict
click
docker
kubernetes>=5.0.0
google-cloud

got:

tornado 4.2 requires backports.ssl-match-hostname, which is not installed.
docker 3.2.1 requires backports.ssl-match-hostname>=3.5; python_version < "3.5", which is not installed.
git-review 1.25.0 requires argparse, which is not installed.
google-cloud-storage 1.6.0 has requirement requests>=2.18.0, but you'll have requests 2.11.1 which is incompatible.
kubernetes 5.0.0 has requirement python-dateutil>=2.5.3, but you'll have python-dateutil 2.4.2 which is incompatible.
kubernetes 5.0.0 has requirement urllib3!=1.21,>=1.19.1, but you'll have urllib3 1.16 which is incompatible.
neptune-cli 2.7.7 has requirement six==1.10.0, but you'll have six 1.11.0 which is incompatible.
google-api-core 0.1.4 has requirement requests<3.0.0dev,>=2.18.0, but you'll have requests 2.11.1 which is incompatible.
google-cloud-vision 0.29.0 has requirement requests<3.0dev,>=2.18.4, but you'll have requests 2.11.1 which is incompatible.
google-cloud-bigquery 0.28.0 has requirement requests>=2.18.0, but you'll have requests 2.11.1 which is incompatible.
google-gax 0.15.16 has requirement requests<3.0dev,>=2.13.0, but you'll have requests 2.11.1 which is incompatible.
docker 3.2.1 has requirement requests!=2.18.0,>=2.14.2, but you'll have requests 2.11.1 which is incompatible.
google-cloud-spanner 0.29.0 has requirement requests<3.0dev,>=2.18.4, but you'll have requests 2.11.1 which is incompatible.

#3

This is of course a bug and we will improve our requirements file to fix this issue.
But in my case this is just a warning and neptune-cli is installed correctly and fully working anyway. Is it also your case? Can you use neptune command after the installation or not? If not, could you provide full logs.

Below are logs from my installation on a new virtualenv:

(venv) anita@anita-fronczak:~/uat_2$ pip install neptune-cli
Collecting neptune-cli
Collecting future==0.16.0 (from neptune-cli)
Collecting more-itertools==4.1.0 (from neptune-cli)
  Using cached more_itertools-4.1.0-py2-none-any.whl
Collecting voluptuous==0.9.3 (from neptune-cli)
Collecting PyYAML==3.12 (from neptune-cli)
Requirement already satisfied: setuptools>=20.3.1 in ./venv/lib/python2.7/site-packages (from neptune-cli) (39.0.1)
Collecting requests-oauthlib==0.8.0 (from neptune-cli)
  Using cached requests_oauthlib-0.8.0-py2.py3-none-any.whl
Collecting tqdm==4.11.2 (from neptune-cli)
  Using cached tqdm-4.11.2-py2.py3-none-any.whl
Collecting kitchen==1.2.4 (from neptune-cli)
Collecting psutil==5.4.3 (from neptune-cli)
Collecting PyJWT==1.5.2 (from neptune-cli)
  Using cached PyJWT-1.5.2-py2.py3-none-any.whl
Collecting raven==6.1.0 (from neptune-cli)
  Using cached raven-6.1.0-py2.py3-none-any.whl
Collecting humanize==0.5.1 (from neptune-cli)
Collecting six==1.10.0 (from neptune-cli)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting certifi>=2017.4.1 (from neptune-cli)
  Using cached certifi-2018.1.18-py2.py3-none-any.whl
Collecting pathlib2==2.3.0 (from neptune-cli)
  Using cached pathlib2-2.3.0-py2.py3-none-any.whl
Collecting mock==2.0.0 (from neptune-cli)
  Using cached mock-2.0.0-py2.py3-none-any.whl
Collecting terminaltables==2.1.0 (from neptune-cli)
Collecting gitdb==0.6.4 (from neptune-cli)
Collecting websocket-client==0.35.0 (from neptune-cli)
Collecting pykwalify==1.5.2 (from neptune-cli)
Collecting Pillow>=1.7.6 (from neptune-cli)
  Using cached Pillow-5.1.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting tornado==4.2 (from neptune-cli)
Collecting GitPython==2.0.8 (from neptune-cli)
Collecting enum34==1.1.6 (from neptune-cli)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting urllib3==1.16 (from neptune-cli)
  Using cached urllib3-1.16-py2.py3-none-any.whl
Collecting Flask==0.12.0 (from neptune-cli)
  Using cached Flask-0.12-py2.py3-none-any.whl
Collecting python-dateutil==2.4.2 (from neptune-cli)
  Using cached python_dateutil-2.4.2-py2.py3-none-any.whl
Collecting git-review==1.25.0 (from neptune-cli)
  Using cached git_review-1.25.0-py2.py3-none-any.whl
Collecting requests==2.11.1 (from neptune-cli)
  Using cached requests-2.11.1-py2.py3-none-any.whl
Collecting oauthlib>=0.6.2 (from requests-oauthlib==0.8.0->neptune-cli)
  Using cached oauthlib-2.0.7-py2.py3-none-any.whl
Collecting contextlib2 (from raven==6.1.0->neptune-cli)
  Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Collecting scandir; python_version < "3.5" (from pathlib2==2.3.0->neptune-cli)
Collecting pbr>=0.11 (from mock==2.0.0->neptune-cli)
  Using cached pbr-4.0.1-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock==2.0.0->neptune-cli)
  Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting smmap>=0.8.5 (from gitdb==0.6.4->neptune-cli)
Collecting docopt>=0.6.2 (from pykwalify==1.5.2->neptune-cli)
Collecting backports.ssl-match-hostname (from tornado==4.2->neptune-cli)
Collecting Jinja2>=2.4 (from Flask==0.12.0->neptune-cli)
  Using cached Jinja2-2.10-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from Flask==0.12.0->neptune-cli)
  Using cached Werkzeug-0.14.1-py2.py3-none-any.whl
Collecting click>=2.0 (from Flask==0.12.0->neptune-cli)
  Using cached click-6.7-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from Flask==0.12.0->neptune-cli)
Requirement already satisfied: argparse in /usr/lib/python2.7 (from git-review==1.25.0->neptune-cli) (1.2.1)
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask==0.12.0->neptune-cli)
tornado 4.2 requires backports.ssl-match-hostname, which is not installed.
git-review 1.25.0 requires argparse, which is not installed.
Installing collected packages: future, six, more-itertools, voluptuous, PyYAML, oauthlib, requests, requests-oauthlib, tqdm, kitchen, psutil, PyJWT, contextlib2, raven, humanize, certifi, scandir, pathlib2, pbr, funcsigs, mock, terminaltables, smmap, gitdb, websocket-client, docopt, python-dateutil, pykwalify, Pillow, backports.ssl-match-hostname, tornado, GitPython, enum34, urllib3, MarkupSafe, Jinja2, Werkzeug, click, itsdangerous, Flask, git-review, neptune-cli
Successfully installed Flask-0.12 GitPython-2.0.8 Jinja2-2.10 MarkupSafe-1.0 Pillow-5.1.0 PyJWT-1.5.2 PyYAML-3.12 Werkzeug-0.14.1 backports.ssl-match-hostname-3.5.0.1 certifi-2018.1.18 click-6.7 contextlib2-0.5.5 docopt-0.6.2 enum34-1.1.6 funcsigs-1.0.2 future-0.16.0 git-review-1.25.0 gitdb-0.6.4 humanize-0.5.1 itsdangerous-0.24 kitchen-1.2.4 mock-2.0.0 more-itertools-4.1.0 neptune-cli-2.7.8 oauthlib-2.0.7 pathlib2-2.3.0 pbr-4.0.1 psutil-5.4.3 pykwalify-1.5.2 python-dateutil-2.4.2 raven-6.1.0 requests-2.11.1 requests-oauthlib-0.8.0 scandir-1.7 six-1.10.0 smmap-0.9.0 terminaltables-2.1.0 tornado-4.2 tqdm-4.11.2 urllib3-1.16 voluptuous-0.9.3 websocket-client-0.35.0

#4

Sure it will work - but other packages may stop working :slight_smile:

If I install neptune-cli as 1st package, then other package - some related to both packages will be upgraded. But if we reverse order - i.e. when neptune-cli will be installed after some other packages, requirements will be downgraded.

As in first post after installation of neptune-cli requests was downgraded from 2.18 to 2.11 and thus google-api-core which require requests<3.0.0dev,>=2.18.0 may fail.

If put some packages with conflicting requirements, I observe that older version of packages are installed (see 2nd post), thus potentially causing errors of packages which require some newer versions.


#5

Sure. Overriding versions of python packages in your environment, can make your code not work and is not very user-friendly. We will prioritize the task, so that we use ranges for python packages.