001. install tensorflow on raspberry pi raspbian buster release 10 (pip)

001. install tensorflow on raspberry pi raspbian buster release 10 (pip)

python3.6 설치하고 삽질만하다가

raspbian buster 로 OS를 새로 설치하고보니 신세계다

python3.7.3 이 기본이고 jupyter notebook 설치하고 에러도 안난다

그러니 tensorflow 도 설치해보기로 했다

 

https://www.tensorflow.org/install/pip

공식홈페이지: https://www.tensorflow.org/install/pip

pip 설치를 권장하고 있다

python3 로 진행해보기로 했다

 

 

필수 사항: Requires Python > 3.4 and pip >= 19.0

1. 필요 lib 설치 1. Install the Python development environment on your system sudo apt update sudo apt install python3-dev python3-pip sudo apt install libatlas-base-dev        # required for numpy sudo pip3 install -U virtualenv           # system-wide install   2. 가상환경 (추천) 2. Create a virtual environment (recommended) virtualenv --system-site-packages -p python3 ./venv   가상환경 실행 source ./venv/bin/activate  # sh, bash, ksh, or zsh   * 가상환경 종료 deactivate  # don't exit until you're done using TensorFlow   3. pip update pip install --upgrade pip pip list  # show packages installed within the virtual environment   4. tensorflow 설치 pip install --upgrade tensorflow   5. tensorflow 테스트 python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

비교적 쉬워보이지만

이전에 용량이 부족했던건지 번번히 tensorflow 설치마다 에러가...

 

* 사양

pi@rasp-dev:~ $ lsb_release -a No LSB modules are available. Distributor ID: Raspbian Description: Raspbian GNU/Linux 10 (buster) Release: 10 Codename: buster   pi@rasp-dev:~ $ uname -a Linux rasp-dev 4.19.93-v7+ #1290 SMP Fri Jan 10 16:39:50 GMT 2020 armv7l GNU/Linux   pi@rasp-dev:~ $ free -h               total        used        free      shared  buff/cache   available Mem:          926Mi       398Mi       343Mi       6.0Mi       183Mi       460Mi Swap:         3.1Gi       142Mi       3.0Gi   pi@rasp-dev:~ $ df -h Filesystem      Size  Used Avail Use% Mounted on /dev/root        15G  3.6G   11G  26% / devtmpfs        459M     0  459M   0% /dev tmpfs           464M  8.0K  464M   1% /dev/shm tmpfs           464M   48M  416M  11% /run tmpfs           5.0M   12K  5.0M   1% /run/lock tmpfs           464M     0  464M   0% /sys/fs/cgroup /dev/mmcblk0p1  253M   54M  199M  22% /boot tmpfs            93M     0   93M   0% /run/user/114 /dev/sda1       3.8G  3.1G  760M  81% /mnt/swap tmpfs            93M     0   93M   0% /run/user/1000   pi@rasp-dev:~/scripts $ pip3 -V pip 20.0.2 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7) pi@rasp-dev:~/scripts $ python3 -V Python 3.7.3

 

1. 필요 lib 설치

sudo apt install python3-dev python3-pip -y

pi@rasp-dev:~ $ sudo apt install python3-dev python3-pip -y Reading package lists... Done Building dependency tree        Reading state information... Done python3-dev is already the newest version (3.7.3-1). python3-pip is already the newest version (18.1-5+rpt1). The following packages were automatically installed and are no longer required:   libminiupnpc17 libnatpmp1 transmission-cli transmission-common Use 'sudo apt autoremove' to remove them. 0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

 

sudo apt install libatlas-base-dev -y

pi@rasp-dev:~ $ sudo apt install libatlas-base-dev Reading package lists... Done Building dependency tree        Reading state information... Done The following packages were automatically installed and are no longer required:   libminiupnpc17 libnatpmp1 transmission-cli transmission-common Use 'sudo apt autoremove' to remove them. The following additional packages will be installed:   libatlas3-base Suggested packages:   libatlas-doc liblapack-doc The following NEW packages will be installed:   libatlas-base-dev libatlas3-base 0 upgraded, 2 newly installed, 0 to remove and 1 not upgraded. Need to get 5,365 kB of archives. After this operation, 32.1 MB of additional disk space will be used. Do you want to continue? [Y/n]  y Get:1 http://ftp.harukasan.org/raspbian/raspbian buster/main armhf libatlas3-base armhf 3.10.3-8+rpi1 [2,399 kB] Get:2 http://ftp.harukasan.org/raspbian/raspbian buster/main armhf libatlas-base-dev armhf 3.10.3-8+rpi1 [2,966 kB] Fetched 5,365 kB in 4s (1,286 kB/s)             Selecting previously unselected package libatlas3-base:armhf. (Reading database ... 71143 files and directories currently installed.) Preparing to unpack .../libatlas3-base_3.10.3-8+rpi1_armhf.deb ... Unpacking libatlas3-base:armhf (3.10.3-8+rpi1) ... Selecting previously unselected package libatlas-base-dev:armhf. Preparing to unpack .../libatlas-base-dev_3.10.3-8+rpi1_armhf.deb ... Unpacking libatlas-base-dev:armhf (3.10.3-8+rpi1) ... Setting up libatlas3-base:armhf (3.10.3-8+rpi1) ... update-alternatives: using /usr/lib/arm-linux-gnueabihf/atlas/libblas.so.3 to provide /usr/lib/arm-linux-gnueabihf/libblas.so.3 (libblas.so.3-arm-linux-gnueabihf) in auto mode update-alternatives: using /usr/lib/arm-linux-gnueabihf/atlas/liblapack.so.3 to provide /usr/lib/arm-linux-gnueabihf/liblapack.so.3 (liblapack.so.3-arm-linux-gnueabihf) in auto mode Setting up libatlas-base-dev:armhf (3.10.3-8+rpi1) ... update-alternatives: using /usr/lib/arm-linux-gnueabihf/atlas/libblas.so to provide /usr/lib/arm-linux-gnueabihf/libblas.so (libblas.so-arm-linux-gnueabihf) in auto mode update-alternatives: using /usr/lib/arm-linux-gnueabihf/atlas/liblapack.so to provide /usr/lib/arm-linux-gnueabihf/liblapack.so (liblapack.so-arm-linux-gnueabihf) in auto mode Processing triggers for libc-bin (2.28-10+rpi1) ...

가상머신 이라고 부르는거 같은데

가상 환경 이라고 불러야 될것 같은...?

 

2. 가상환경 (추천)

가상화 툴 설치

sudo pip3 install -U virtualenv

pi@rasp-dev:~ $ sudo pip3 install -U virtualenv Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Collecting virtualenv   Downloading virtualenv-16.7.9-py2.py3-none-any.whl (3.4 MB)      |████████████████████████████████| 3.4 MB 1.4 MB/s  Installing collected packages: virtualenv Successfully installed virtualenv-16.7.9

 

python3 가상환경 만들기

python_env 디렉토리를 하나 만들었다

virtualenv --system-site-packages -p python3 ./venv

pi@rasp-dev:~/python_env $ virtualenv --system-site-packages -p python3 ./venv Already using interpreter /usr/bin/python3 Using base prefix '/usr' New python executable in /mnt/db/python_env/venv/bin/python3 Also creating executable in /mnt/db/python_env/venv/bin/python Installing setuptools, pip, wheel... done.

 

가상환경 활성화

source ./venv/bin/activate

pi@rasp-dev:~/python_env $ source ./venv/bin/activate (venv) pi@rasp-dev:~/python_env $

종료는 deactivate

 

3. 가상환경 pip 업데이트

pip install --upgrade pip

(venv) pi@rasp-dev:~/python_env $ pip install --upgrade pip Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Requirement already up-to-date: pip in ./venv/lib/python3.7/site-packages (20.0.2) (venv) pi@rasp-dev:~/python_env $

 

(venv) pi@rasp-dev:~/python_env $ python -V Python 3.7.3 (venv) pi@rasp-dev:~/python_env $ pip -V pip 20.0.2 from /home/pi/python_env/venv/lib/python3.7/site-packages/pip (python 3.7)

참고로 python, pip 3.7.3 으로 시스템걸 복사했으니 당연할지도

 

가상환경에 설치된 lib 목록 조회

pip list

(venv) pi@rasp-dev:~/python_env $ pip list Package            Version   ------------------ --------- asn1crypto         0.24.0    attrs              19.3.0    backcall           0.1.0     bleach             3.1.0     certifi            2018.8.24 chardet            3.0.4     cryptography       2.6.1     cycler             0.10.0    decorator          4.3.0     defusedxml         0.6.0     entrypoints        0.3       enum34             1.1.6     idna               2.6       importlib-metadata 1.4.0     ipykernel          5.1.3     ipython            7.11.1    ipython-genutils   0.2.0     ipywidgets         7.5.1     jedi               0.16.0    Jinja2             2.10.3    jsonpickle         1.2       jsonschema         3.2.0     jupyter            1.0.0     jupyter-client     5.3.4     jupyter-console    6.1.0     jupyter-core       4.6.1     keyring            17.1.1    keyrings.alt       3.1.1     kiwisolver         1.0.1     MarkupSafe         1.1.1     matplotlib         3.0.2     mistune            0.8.4     myDevices          2.0.3     nbconvert          5.6.1     nbformat           5.0.4     netifaces          0.10.9    notebook           6.0.3     numpy              1.16.2    olefile            0.46      paho-mqtt          1.5.0     pandocfilters      1.4.2     parso              0.5.2     pexpect            4.8.0     pickleshare        0.7.5     Pillow             5.4.1     pip                20.0.2    prometheus-client  0.7.1     prompt-toolkit     3.0.2     psutil             5.6.7     ptyprocess         0.6.0     pycrypto           2.6.1     Pygments           2.5.2     PyGObject          3.30.4    pyparsing          2.2.0     pyrsistent         0.15.7    python-apt         1.8.4.1   python-dateutil    2.7.3     pyxdg              0.25      pyzmq              18.1.1    qtconsole          4.6.0     requests           2.21.0    scipy              1.1.0     SecretStorage      2.3.1     Send2Trash         1.5.0     setuptools         45.1.0    six                1.12.0    ssh-import-id      5.7       terminado          0.8.3     testpath           0.4.4     tornado            6.0.3     traitlets          4.3.3     urllib3            1.24.1    virtualenv         16.7.9    wcwidth            0.1.8     webencodings       0.5.1     wheel              0.33.6    widgetsnbextension 3.5.1     zipp               2.1.0

 

4. tensorflow 설치

pip install --upgrade tensorflow

(venv) pi@rasp-dev:~/python_env $ pip install --upgrade tensorflow Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Collecting tensorflow   Downloading https://www.piwheels.org/simple/tensorflow/tensorflow-1.14.0-cp37-none-linux_armv7l.whl (79.6 MB)      |████████████████████████████████| 79.6 MB 4.5 kB/s  Requirement already satisfied, skipping upgrade: six>=1.10.0 in /usr/lib/python3/dist-packages (from tensorflow) (1.12.0) WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /simple/google-pasta/ Collecting google-pasta>=0.1.6   Downloading google_pasta-0.1.8-py3-none-any.whl (57 kB)      |████████████████████████████████| 57 kB 125 kB/s  Collecting tensorboard<2.1.0,>=2.0.0   Downloading tensorboard-2.0.2-py3-none-any.whl (3.8 MB)      |████████████████████████████████| 3.8 MB 5.1 MB/s  Requirement already satisfied, skipping upgrade: numpy<2.0,>=1.16.0 in /usr/lib/python3/dist-packages (from tensorflow) (1.16.2) Collecting grpcio>=1.8.6   Downloading https://www.piwheels.org/simple/grpcio/grpcio-1.26.0-cp37-cp37m-linux_armv7l.whl (17.2 MB)      |████████████████████████████████| 17.2 MB 10 kB/s  Requirement already satisfied, skipping upgrade: wheel>=0.26; python_version >= "3" in ./venv/lib/python3.7/site-packages (from tensorflow) (0.33.6) WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /simple/astor/ Collecting astor>=0.6.0   WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /packages/c3/88/97eef84f48fa04fbd6750e62dcceafba6c63c81b7ac1420856c8dcc0a3f9/astor-0.8.1-py2.py3-none-any.whl   Downloading astor-0.8.1-py2.py3-none-any.whl (27 kB) Collecting protobuf>=3.6.1   Downloading protobuf-3.11.2-py2.py3-none-any.whl (434 kB)      |████████████████████████████████| 434 kB 353 kB/s  Collecting termcolor>=1.1.0   Downloading https://www.piwheels.org/simple/termcolor/termcolor-1.1.0-py3-none-any.whl (4.8 kB) Collecting absl-py>=0.7.0   Downloading https://www.piwheels.org/simple/absl-py/absl_py-0.9.0-py3-none-any.whl (121 kB)      |████████████████████████████████| 121 kB 44 kB/s  Collecting opt-einsum>=2.3.2   Downloading https://www.piwheels.org/simple/opt-einsum/opt_einsum-3.1.0-py3-none-any.whl (63 kB)      |████████████████████████████████| 63 kB 24 kB/s  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/gast/ Collecting gast==0.2.2   Downloading https://www.piwheels.org/simple/gast/gast-0.2.2-py3-none-any.whl (7.5 kB) Collecting keras-preprocessing>=1.0.5   Downloading Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41 kB)      |████████████████████████████████| 41 kB 110 kB/s  Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0   Downloading tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488 kB)      |████████████████████████████████| 488 kB 3.3 MB/s  Collecting keras-applications>=1.0.8   Downloading Keras_Applications-1.0.8-py3-none-any.whl (50 kB)      |████████████████████████████████| 50 kB 885 kB/s  Collecting wrapt>=1.11.1   Downloading https://www.piwheels.org/simple/wrapt/wrapt-1.11.2-cp37-cp37m-linux_armv7l.whl (68 kB)      |████████████████████████████████| 68 kB 22 kB/s  Collecting google-auth-oauthlib<0.5,>=0.4.1   Downloading google_auth_oauthlib-0.4.1-py2.py3-none-any.whl (18 kB) Collecting markdown>=2.6.8   Downloading Markdown-3.1.1-py2.py3-none-any.whl (87 kB)      |████████████████████████████████| 87 kB 494 kB/s  Requirement already satisfied, skipping upgrade: setuptools>=41.0.0 in ./venv/lib/python3.7/site-packages (from tensorboard<2.1.0,>=2.0.0->tensorflow) (45.1.0) WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/werkzeug/ Collecting werkzeug>=0.11.15   Downloading Werkzeug-0.16.0-py2.py3-none-any.whl (327 kB)      |████████████████████████████████| 327 kB 7.0 MB/s  Requirement already satisfied, skipping upgrade: requests<3,>=2.21.0 in /usr/lib/python3/dist-packages (from tensorboard<2.1.0,>=2.0.0->tensorflow) (2.21.0) Collecting google-auth<2,>=1.6.3   Downloading google_auth-1.11.0-py2.py3-none-any.whl (76 kB)      |████████████████████████████████| 76 kB 758 kB/s  Collecting h5py   Downloading https://www.piwheels.org/simple/h5py/h5py-2.10.0-cp37-cp37m-linux_armv7l.whl (4.7 MB)      |████████████████████████████████| 4.7 MB 12 kB/s  Collecting requests-oauthlib>=0.7.0   Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB) Collecting pyasn1-modules>=0.2.1   Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)      |████████████████████████████████| 155 kB 6.7 MB/s  Collecting rsa<4.1,>=3.1.4   Downloading rsa-4.0-py2.py3-none-any.whl (38 kB) Collecting cachetools<5.0,>=2.0.0   Downloading cachetools-4.0.0-py3-none-any.whl (10 kB) Collecting oauthlib>=3.0.0   Downloading oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)      |████████████████████████████████| 147 kB 7.4 MB/s  Collecting pyasn1<0.5.0,>=0.4.6   Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)      |████████████████████████████████| 77 kB 1.8 MB/s  Installing collected packages: google-pasta, oauthlib, requests-oauthlib, pyasn1, pyasn1-modules, rsa, cachetools, google-auth, google-auth-oauthlib, markdown, protobuf, absl-py, grpcio, werkzeug, tensorboard, astor, termcolor, opt-einsum, gast, keras-preprocessing, tensorflow-estimator, h5py, keras-applications, wrapt, tensorflow Successfully installed absl-py-0.9.0 astor-0.8.1 cachetools-4.0.0 gast-0.2.2 google-auth-1.11.0 google-auth-oauthlib-0.4.1 google-pasta-0.1.8 grpcio-1.26.0 h5py-2.10.0 keras-applications-1.0.8 keras-preprocessing-1.1.0 markdown-3.1.1 oauthlib-3.1.0 opt-einsum-3.1.0 protobuf-3.11.2 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-oauthlib-1.3.0 rsa-4.0 tensorboard-2.0.2 tensorflow-1.14.0 tensorflow-estimator-1.14.0 termcolor-1.1.0 werkzeug-0.16.0 wrapt-1.11.2

 

오 성공

2시간쯤 걸린듯..

 

5. tensorflow 테스트

python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

(venv) pi@rasp-dev:~/python_env $ python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))" 2020-01-27 01:08:32.110370: E tensorflow/core/platform/hadoop/hadoop_file_system.cc:132] HadoopFileSystem load error: libhdfs.so: cannot open shared object file: No such file or directory Tensor("Sum:0", shape=(), dtype=float32)   (venv) pi@rasp-dev:~/python_env $ python Python 3.7.3 (default, Apr  3 2019, 05:39:12)  [GCC 8.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import tensorflow as tf 2020-01-27 01:09:29.853174: E tensorflow/core/platform/hadoop/hadoop_file_system.cc:132] HadoopFileSystem load error: libhdfs.so: cannot open shared object file: No such file or directory >>> exit()

성공? 실패?

hadoop libhdfs.so 을 찾다가 No such file or directory 를 뿜어주신다

근데 module load에 실패 한건아니니.. github에서도 논쟁으로 보이는데

무시하고 print(tf) 를 해보면 결과가 나온다

 

hadoop 문제는 시간날때 찾아봐야겠다

 

참고

https://www.tensorflow.org/install/pip