不知道是不是因为在这边华人算少数，感觉现在听歌都是听的中文甚至是老歌，有多老呢，SHE，都追溯到小学时代了。还有林俊杰，以前我主要是对他那首杀手印象很深，旋律好听歌词变态，上次有人给我推荐了他的新歌光阴副本，然后发现自己莫名其妙循环了很久Turn Of A Page。还有I Sent My Therapist to Therapy，今年四月一号还是二号Alec Benjamin来哥哈开演唱会，我好像是提前了半个月还是多久发现的，剩下的票太贵，一千多虽然也不是不行，但想想那段时间好忙，算了。可是没有最忙只有更忙，这几个月比三四月份癫多了
\n
M前几天回来和他吃饭，他问我暑假回国没，我说提前六月初回了，他问我啥感受，我说“好吃”，他😂……不是我说，哥哈的中餐馆有一个算一个，没几个能打的，放国内过阵子都得倒闭吧，而且有个规律，刚开业时候还行的过段时间就不行了，据有个dk同事说这是厨师们到处跳槽涨工资，good to know，原来靠这涨身价的不止码农。我现在只能靠泰餐改善生活，食堂叫Rabbit名副其实都是喂Rabbit的 (ironically, 连兔子都不是只吃素的)。去年我从新加坡回去时候跟同学吃饭，每吃一个都说好吃，我同学说“你怎么跟逃难的回来了似的”，现在北欧宁古塔比那时还不如，我都被逼得开始自己做饭了，当然也有四月份餐馆都涨了一波价的缘故，一个可颂都动辄三四十还是便宜的，哥哈真是太贵了，吃不起……
PyCharm Community Edition: C:\\Program Files\\JetBrains\\PyCharm Community Edition 2020.3\\bin;
\n
TEMP: C:\\Users\\Lenovo\\AppData\\Local\\Temp
\n
TMP: C:\\Users\\Lenovo\\AppData\\Local\\Temp
\n
\n
\n
\n
查看CUDA版本
1 2 3 4 5 6 7
λ nvcc --version λ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2019 NVIDIA Corporation Built on Sun_Jul_28_19:12:52_Pacific_Daylight_Time_2019 Cuda compilation tools, release 10.1, V10.1.243
>>> import tensorflow as tf C:\\Users\\Lenovo\\Anaconda3\\lib\\site-packages\\numpy\\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs: C:\\Users\\Lenovo\\Anaconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.NOIJJG62EMASZI6NYURL6JBKM4EVBGM7.gfortran-win_amd64.dll C:\\Users\\Lenovo\\Anaconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll stacklevel=1) 2021-01-05 13:24:03.251532: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_100.dll'; dlerror: cudart64_100.dll not found >>> tf.__version__ '2.0.0' >>> ^Z
\n
\n
\n
tensorflow-gpu 2.1.0
\n
Python 3.7.0 (Anaconda3 5.3.1 64-bit)
\n
CUDA 10.1 + cuDNN 7.6.5
\n
tensorboard 2.1.1
\n
tensorflow-gpu 2.1.0
\n
tensorflow-gpu-estimator 2.1.0
1 2 3 4 5 6 7
>>> import tensorflow as tf 2021-01-05 16:28:30.524528: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll >>> tf.test.is_gpu_available() WARNING:tensorflow:From <stdin>:1: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.config.list_physical_devices('GPU')` instead. ....
$ cd GitHubLab # ls $ vim ILSVRC2012.sh $ chmod +x ILSVRC2012.sh $ ./ILSVRC2012.sh $ vim proprocess_imagenet_validation_data.py $ vim build_imagenet_data.py
C = Compute, which defines the processes that use the compute mode of Nvidia GPUs which use CUDA libraries, used in deep learning training and inferencing using Tensorflow-GPU, Pytorch, etc
\n
G = Graphics, which defines the processes that use the graphics mode of Nvidia GPUs used by professional 3D graphics, gnome-shell (Ubuntu’s GUI environment), Games, etc for the rendering of graphics or videos
\n
C+G = Compute + Graphics, which defines the processes that use both the contexts defined above.
$ axel -a -n 10 <url-of-the-file-to-be-downloaded>
\n
使用参数
\n
一般使用：axel url（下载文件地址）；
\n
限速使用：加上 -s 参数，如 -s 10240，即每秒下载的字节数，这里是 10 Kb；
\n
限制连接数：加上 -n 参数，如 -n 10，即打开10个连接。
\n
\n
\n
更改默认线程数
\n
设置线程数为32线程，或者设置成更多（默认为10线程）
1
vim /usr/lib/yum-plugins/axelget.py
\n
修改如下，将10改为32，两个位置：\n\n
\n
\n
\n
\n
下载
1 2 3
$ cd ~/Software $ axel -a -n 10 https://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.243_418.87.00_linux.run $ axel -a -n 10 https://developer.nvidia.com/compute/machine-learning/cudnn/secure/7.6.5.32/Production/10.1_20191031/cudnn-10.1-linux-x64-v7.6.5.32.tgz
CUDA Driver │ [ ] Do not install any of the OpenGL-related driver files │ │ [ ] Do not install the nvidia-drm kernel module │ │ [ ] Update the system X config file to use the NVIDIA X driver │ │ Change directory containing the kernel source files
CUDA Toolkit Change Toolkit Install Path: /home/eustomaqua/Software/cuda-10.1 [ ] Create symbolic link from /usr/local/cuda - [ ] Create desktop menu shortcuts [X] Yes [ ] Install manpage documents to /usr/share/man
Library install path (Blank for system default) # /home/eustomaqua/Software/cuda-library-10.1 # Leave it as blank (as it should be) # Library install path: /home/eustomaqua/Software/cuda-10.1 $
Driver: Not Selected Toolkit: Installed in /home/eustomaqua/Software/cuda-10.1/ Samples: Installed in /home/eustomaqua/Software/cuda-samples-101/, but missing recommended libraries
Please make sure that - PATH includes /home/eustomaqua/Software/cuda-10.1/bin - LD_LIBRARY_PATH includes /home/eustomaqua/Software/cuda-10.1/lib64, or, add /home/eustomaqua/Software/cuda-10.1/lib64 to /etc/ld.so.conf and run ldconfig as root
To uninstall the CUDA Toolkit, run cuda-uninstaller in /home/eustomaqua/Software/cuda-10.1/bin
Please see CUDA_Installation_Guide_Linux.pdf in /home/eustomaqua/Software/cuda-10.1/doc/pdf for detailed information on setting up CUDA. ***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 418.00 is required for CUDA 10.1 functionality to work. To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file: sudo <CudaInstaller>.run --silent --driver
Logfile is /tmp/cuda-installer.log eustomaqua@loae-ws:~/Software$
Driver: Not Selected Toolkit: Installed in /home/yjbian/Software/cuda-10.1/ Samples: Installed in /home/yjbian/Software/cuda-samples-101/, but missing recommended libraries
Please make sure that - PATH includes /home/yjbian/Software/cuda-10.1/bin - LD_LIBRARY_PATH includes /home/yjbian/Software/cuda-10.1/lib64, or, add /home/yjbian/Software/cuda-10.1/lib64 to /etc/ld.so.conf and run ldconfig as root
To uninstall the CUDA Toolkit, run cuda-uninstaller in /home/yjbian/Software/cuda-10.1/bin
Please see CUDA_Installation_Guide_Linux.pdf in /home/yjbian/Software/cuda-10.1/doc/pdf for detailed information on setting up CUDA. ***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 418.00 is required for CUDA 10.1 functionality to work. To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file: sudo <CudaInstaller>.run --silent --driver
$ cd Software $ sudorm -r cuda-10.1 cuda-samples-101 $ ./cuda_10.1.105_418.39_linux.run
Options: CUDA Toolkit Change Toolkit Install Path: /home/eustomaqua/Software/cuda-10.1/ [ ] Create symbolic link from /usr/local/cuda - [X] Create desktop menu shortcuts [ ] All users [X] Yes [ ] No [ ] Install manpage documents to /usr/share/man CUDA Samples Change Sample Install Path: /home/eustomaqua/Software/cuda-samples-101/ Root install path: /home/eustomaqua/Software/cuda-10.1/
=========== = Summary = ===========
Driver: Not Selected Toolkit: Installed in /home/yjbian/Software/cuda-10.1/ Samples: Installed in /home/yjbian/Software/cuda-samples-101/, but missing recommended libraries
Please make sure that - PATH includes /home/yjbian/Software/cuda-10.1/bin - LD_LIBRARY_PATH includes /home/yjbian/Software/cuda-10.1/lib64, or, add /home/yjbian/Software/cuda-10.1/lib64 to /etc/ld.so.conf and run ldconfig as root
To uninstall the CUDA Toolkit, run cuda-uninstaller in /home/yjbian/Software/cuda-10.1/bin
Please see CUDA_Installation_Guide_Linux.pdf in /home/yjbian/Software/cuda-10.1/doc/pdf for detailed information on setting up CUDA. ***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 418.00 is required for CUDA 10.1 functionality to work. To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file: sudo <CudaInstaller>.run --silent --driver
Logfile is /tmp/cuda-installer.log $ vim ~/.bashrc $ tar -xvf cudnn-10.1-linux-x64-v7.6.5.32.tgz $ mv cuda/include/cudnn.h ~/Software/cuda-10.1/include/ $ mv cuda/lib64/libcudnn* ~/Software/cuda-10.1/lib64/ $ chmod a+r ~/Software/cuda-10.1/include/cudnn.h ~/Software/cuda-10.1/lib64/libcudnn* $ rm -r cuda $ rm -r TensorRT-6.0.1.5 $ tar -xvf TensorRT-6.0.1.5.Ubuntu-18.04.x86_64-gnu.cuda-10.1.cudnn7.6.tar.gz $ cd TensorRT-6.0.1.5 $ cd python $ pip install tensorrt-6.0.1.5-cp36-none-linux_x86_64.whl $ cd ../uff $ pip install uff-0.6.5-py2.py3-none-any.whl $ cd ~/anaconda3/envs/adapruning/ $ cd ~/anaconda3/lib/python3.6/site-packages $ cd ~/anaconda3/envs/adapruning/lib/python3.6/site-packages $ cd uff/converters/tensorflow/ $ vim conversion_helpers.py # from tensorflow import GraphDef from tensorflow.compat.v1 import GraphDef $ python >>> import tensorrt >>> import uff >>> import tensorflow as tf >>> tf.test.is_gpu_available() >>> import tensorflow.compat.v1 as tf >>> tf.test.is_gpu_available()
# http://landcareweb.com/questions/26327/ru-he-jiang-tensorflowri-zhi-ji-lu-zhong-ding-xiang-dao-wen-jian # get TF logger log = logging.getLogger('tensorflow') log.setLevel(logging.DEBUG) # create formatter and add it to the handlers #formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') # create file handler which logs even debug messages fh = logging.FileHandler('tensorflow.log') fh.setLevel(logging.DEBUG) #fh.setFormatter(formatter) log.addHandler(fh)
\n\n
TensorBoard
Same Terminal, Different Machines
\n
1 2 3 4 5 6 7 8 9
# Local $ ssh -L 16006:127.0.0.1:6006 yourname@server.address # Server $ cd expected-folder-path $ tensorboard --logdir=./network $ # Open http://127.0.0.1:16006 in Local browser $ # Ctrl+C close on the same Terminal
$ cd ~ $ mv software Software $ cd ~/Software $ bash ./Anaconda3-5.2.0-Linux-x86_64.sh
In order to continue the installation process, please review the license agreement. Please, press ENTER to continue >>>
Do you accept the license terms? [yes|no] [no] >>> Please answer 'yes' or 'no':' >>> yes Anaconda3 will now be installed into this location: /home/eustomaqua/anaconda3 - Press ENTER to confirm the location - Press CTRL-C to abort the installation - Or specify a different location below [/home/eustomaqua/anaconda3] >>> /home/eustomaqua/VirtualEnv/anaconda3 PREFIX=/home/eustomaqua/VirtualEnv/anaconda3 installing: python-3.6.5-hc3d631a_2 ... Python 3.6.5 :: Anaconda, Inc. installing: blas-1.0-mkl ... installation finished. WARNING: You currently have a PYTHONPATH environment variable set. This may cause unexpected behavior when running the Python interpreter in Anaconda3. For best results, please verify that your PYTHONPATH only points to directories of packages that are compatible with the Python interpreter in Anaconda3: /home/eustomaqua/VirtualEnv/anaconda3 Do you wish the installer to prepend the Anaconda3 install location to PATH in your /home/eustomaqua/.bashrc ? [yes|no] [no] >>> yes Appending source /home/eustomaqua/VirtualEnv/anaconda3/bin/activate to /home/eustomaqua/.bashrc A backup will be made to: /home/eustomaqua/.bashrc-anaconda3.bak For this change to become active, you have to open a new terminal. Thank you for installing Anaconda3! =========================================================================== Anaconda is partnered with Microsoft! Microsoft VSCode is a streamlined code editor with support for development operations like debugging, task running and version control. To install Visual Studio Code, you will need: - Administrator Privileges - Internet connectivity Visual Studio Code License: https://code.visualstudio.com/license Do you wish to proceed with the installation of Microsoft VSCode? [yes|no] >>> no $
\n\n
检查
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
~/software$ python Python 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license"for more information. >>> [1]+ Stopped(SIGTSTP) python ~/software$
~/software$ source ~/.bashrc ~/software$ python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> [2]+ Stopped(SIGTSTP) python ~/software$ ~/software$ pip list
~$ cd Software ~/Software$ ls Anaconda3-5.2.0-Linux-x86_64.sh cudnn-10.0-linux-x64-v7.6.4.38.solitairetheme8 cuda_10.0.130_410.48_linux.run Python-3.6.1.tgz
~/Software$ chmod +x cuda_10.0.130_410.48_linux.run ~/Software$ ls Anaconda3-5.2.0-Linux-x86_64.sh cudnn-10.0-linux-x64-v7.6.4.38.solitairetheme8 cuda_10.0.130_410.48_linux.run Python-3.6.1.tgz
~/Software$ ./cuda_10.0.130_410.48_linux.run Logging to /tmp/cuda_install_25104.log Using more to view the EULA. End User License Agreement --------------------------
20. Licensee's use of linmath.h header for CPU functions for GL vector/matrix operations from lunarG is subject to the Apache License Version 2.0. ----------------- Do you accept the previously read EULA? accept/decline/quit: Do you accept the previously read EULA? accept/decline/quit: accept Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 410.48? (y)es/(n)o/(q)uit: no Install the CUDA 10.0 Toolkit? (y)es/(n)o/(q)uit: yes Enter Toolkit Location [ default is /usr/local/cuda-10.0 ]: /home/eustomaqua/Software/cuda-10.0 Do you want to install a symbolic link at /usr/local/cuda? (y)es/(n)o/(q)uit: no Install the CUDA 10.0 Samples? (y)es/(n)o/(q)uit: yes Enter CUDA Samples Location [ default is /home/eustomaqua ]: /home/eustomaqua/Software/cuda-samples $
~/Software$ ls Anaconda3-5.2.0-Linux-x86_64.sh cuda-samples cuda cudnn-10.0-linux-x64-v7.6.4.38.solitairetheme8 cuda-10.0 cudnn-10.0-linux-x64-v7.6.4.38.tgz cuda_10.0.130_410.48_linux.run Python-3.6.1.tgz ~/Software$ rm cudnn-10.0-linux-x64-v7.6.4.38.solitairetheme8 ~/Software$ cd cuda :~/Software/cuda$ ls include lib64 NVIDIA_SLA_cuDNN_Support.txt ~/Software/cuda$ cd ..
在 Nvidia 官网注册，填写调查问卷之后可下载 TensorRT NVIDIA TensorRT Click Download Now 下载前先检查下自己的开发环境 —— 系统和 CUDA 版本
\n
\n
~$ cat /etc/issue\nUbuntu 18.04.3 LTS \\n \\l\n\n~$ cat /usr/local/cuda/version.txt\ncat: /usr/local/cuda/version.txt: No such file or directory\n~$ cat ~/Software/cuda-10.0/version.txt\nCUDA Version 10.0.130\n~$\n
1 2 3 4 5 6 7 8 9
- ```bash ~/Software$ cat /etc/issue Ubuntu 16.04.6 LTS \\n \\l ~/Software$ cat /usr/local/cuda/version.txt CUDA Version 10.2.89 ~/Software$ cat ~/Software/cuda-10.0/version.txt CUDA Version 10.0.130 ~/Software$
~/Software$ # 3. 进入解压后的 TensorRT 目录下的 python 文件夹 ~/Software$ cd TensorRT-7.0.0.11 ~/Software/TensorRT-7.0.0.11$ ls bin doc include python targets uff data graphsurgeon lib samples TensorRT-Release-Notes.pdf ~/Software/TensorRT-7.0.0.11$ cd Python -sh: cd: Python: No such file or directory ~/Software/TensorRT-7.0.0.11$ cd python ~/Software/TensorRT-7.0.0.11/python$ ls tensorrt-7.0.0.11-cp27-none-linux_x86_64.whl tensorrt-7.0.0.11-cp36-none-linux_x86_64.whl tensorrt-7.0.0.11-cp34-none-linux_x86_64.whl tensorrt-7.0.0.11-cp37-none-linux_x86_64.whl tensorrt-7.0.0.11-cp35-none-linux_x86_64.whl ~/Software/TensorRT-7.0.0.11/python$
Python 3.6.10 |Anaconda, Inc.| (default, Mar 252020, 23:51:54) [GCC 7.3.0] on linux Type"help", "copyright", "credits"or"license"for more information. >>> import tensorflow as tf 2020-04-03 07:02:35.550509: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6 2020-04-03 07:02:35.552571: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6 >>> import adanet WARNING:tensorflow:From /home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/core/tpu_estimator.py:39: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead.
>>> >>> import tensorrt >>> import uff Traceback (most recent call last): File "/home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/uff/converters/tensorflow/conversion_helpers.py", line 18, in <module> from tensorflow import GraphDef ImportError: cannot import name 'GraphDef'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/uff/__init__.py", line 2, in <module> from uff.converters.tensorflow.conversion_helpers import from_tensorflow # noqa File "/home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/uff/converters/tensorflow/conversion_helpers.py", line 23, in <module> https://www.tensorflow.org/install/""".format(err)) ImportError: ERROR: Failed to import module (cannot import name 'GraphDef') Please make sure you have TensorFlow installed. For installation instructions, see: https://www.tensorflow.org/install/ >>>
It seems to be due to Tensorflow version. UFF converter not supporting TF version 2.0. Please check the tensorflow version and install tensorflow version 1.15 or 1.14.
Python 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorrt Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'tensorrt' >>> import tensorflow as tf >>> import adanet WARNING:tensorflow:From /home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/tf_compat/__init__.py:87: The name tf.losses.Reduction is deprecated. Please use tf.compat.v1.losses.Reduction instead. WARNING:tensorflow:From /home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/core/tpu_estimator.py:39: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead. >>>
$ cd ~/Software/TensorRT-6.0.1.5 $ cd python $ pip install tensorrt-6.0.1.5-cp36-none-linux_x86_64.whl $ cd ../uff $ pip install uff-0.6.5-py2.py3-none-any.whl
Python 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorrt >>> import uff >>> import tensorflow as tf >>> import adanet WARNING:tensorflow:From /home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/tf_compat/__init__.py:87: The name tf.losses.Reduction is deprecated. Please use tf.compat.v1.losses.Reduction instead.
WARNING:tensorflow:From /home/eustomaqua/VirtualEnv/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/core/tpu_estimator.py:39: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead.
Python 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorflow as tf >>> import adanet WARNING:tensorflow:From /home/eustomaqua/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/tf_compat/__init__.py:87: The name tf.losses.Reduction is deprecated. Please use tf.compat.v1.losses.Reduction instead. WARNING:tensorflow:From /home/eustomaqua/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/core/tpu_estimator.py:39: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead. >>> [1]+ Stopped python
\n\n
安装 TensorRT
\n
1 2 3 4 5 6 7 8 9 10 11
$ cd ~/Software $ ls cuda-10.0 cuda-samples nv-tensorrt-repo-ubuntu1804-cuda10.0-trt6.0.1.5-ga-20190913_1-1_amd64.deb nv-tensorrt-repo-ubuntu1804-cuda10.0-trt7.0.0.11-ga-20191216_1-1_amd64.deb TensorRT-6.0.1.5.Ubuntu-18.04.x86_64-gnu.cuda-10.0.cudnn7.6.tar.gz TensorRT-7.0.0.11.Ubuntu-18.04.x86_64-gnu.cuda-10.0.cudnn7.6.tar.gz
$ cd ~/Software $ tar -xvaf TensorRT-6.0.1.5.Ubuntu-18.04.x86_64-gnu.cuda-10.0.cudnn7.6.tar.gz $ cd TensorRT-6.0.1.5 $ cd python $ pip install tensorrt-6.0.1.5-cp36-none-linux_x86_64.whl $ cd ../uff $ pip install uff-0.6.5-py2.py3-none-any.whl
$ source activate adanet $ cd ~/Software/TensorRT-6.0.1.5
Python 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorrt >>> import uff >>> import tensorflow as tf >>> import adanet WARNING:tensorflow:From /home/eustomaqua/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/tf_compat/__init__.py:87: The name tf.losses.Reduction is deprecated. Please use tf.compat.v1.losses.Reduction instead.
WARNING:tensorflow:From /home/eustomaqua/anaconda3/envs/adanet/lib/python3.6/site-packages/adanet/core/tpu_estimator.py:39: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead.
$ cd ~/Software/TensorRT-6.0.1.5 $ which convert-to-uff /home/eustomaqua/anaconda3/envs/adanet/bin/convert-to-uff $ $ mkdir python/data $ mkdir python/data/mnist $ cp ./data/mnist/lenet5.uff ./python/data/mnist/lenet5.uff $ cd ./samples/sampleMNIST $ make clean $ make $ cd ../../bin $ ./sample_mnist &&&& RUNNING TensorRT.sample_mnist # ./sample_mnist [03/03/2020-07:47:01] [I] Building and running a GPU inference engine for MNIST [03/03/2020-07:47:07] [I] [TRT] Detected 1 inputs and 1 output network tensors. Could not find 8.pgm in data directories: data/mnist/ data/samples/mnist/ &&&& FAILED
# 1. Install CUDA 10.0.130, cuDNN 7.6.4 # install tensorflow-gpu=2.1.0 or 1.15.2 (recommend 1.15.2 for uff) $ pip install tensorflow-gpu==1.*
# 2. download TensorRT tar from NVIDIA $ wget TensorRT-6.0.1.5.Ubuntu-{16/18}.04.x86_64-gnu.cuda-10.0.cudnn7.6.tar.gz $ tar -xzvf TensorRT-6.0.1.5.Ubuntu-16.04.x86_64-gpu.cuda-10.0.cudnn7.6.tar.gz
# 3. .bashrc $ vim ~/.bashrc
\n
1 2 3 4 5 6 7 8 9
# add by Anaconda3 install export PATH="/home/eustomaqua/VirtualEnv/anaconda3/bin:$PATH" # self added for Nvidia export PATH=$HOME/Software/cuda-10.0/bin:$PATH export LD_LIBRARY_PATH=$HOME/Software/cuda-10.0/lib64:$LD_LIBRARY_PATH # self modified for TensorRT export LD_LIBRARY_PATH=$HOME/Software/TensorRT-6.0.1.5/lib:$LD_LIBRARY_PATH export CUDA_INSTALL_DIR=$HOME/Software/cuda-10.0 export CUDNN_INSTALL_DIR=$HOME/Software/cuda-10.0
\n
1 2 3 4 5 6 7 8 9 10 11
$ # source ~/.bashrc $ echo$HOME
# 4. install tensorrt $ cd TensorRT-6.0.1.5 $ cd python $ pip install tensorrt-6.0.1.5-cp36-none-linux_x86_64.whl
\n","categories":["Records"],"tags":["Configuration","Linux","NVIDIA"]},{"title":"Setup on Linux (Ubuntu, CentOS 7)","url":"https://eustomaqua.github.io/2020/2020-02-08-Setup-Python-on-Linux/","content":"\n\n\n
[TOC]
\n
1 2 3 4 5
$ cd ~ $ ls $ $ mkdir Software $ # upload Anaconda*.sh
In order to continue the installation process, please review the license agreement. Please, press ENTER to continue >>>
>>> Please answer 'yes' or 'no':' >>> yes Anaconda3 will now be installed into this location: /home/eustomaqua/anaconda3 - Press ENTER to confirm the location - Press CTRL-C to abort the installation - Or specify a different location below [/home/eustomaqua/anaconda3] >>> '/home/eustomaqua/Software/anaconda3 ./Anaconda3-5.2.0-Linux-x86_64.sh: eval: line 301: unexpected EOF while looking for matching `'' ./Anaconda3-5.2.0-Linux-x86_64.sh: eval: line 302: syntax error: unexpected end of file PREFIX=/home/eustomaqua/anaconda3 installing: python-3.6.5-hc3d631a_2 ... Python 3.6.5 :: Anaconda, Inc. installing: blas-1.0-mkl ...
installing: anaconda-5.2.0-py36_3 ... installation finished. Do you wish the installer to prepend the Anaconda3 install location to PATH in your /home/eustomaqua/.bashrc ? [yes|no] [no] >>> yes
Appending source /home/eustomaqua/anaconda3/bin/activate to /home/eustomaqua/.bashrc A backup will be made to: /home/eustomaqua/.bashrc-anaconda3.bak
For this change to become active, you have to open a new terminal.
Anaconda is partnered with Microsoft! Microsoft VSCode is a streamlined code editor with support for development operations like debugging, task running and version control.
To install Visual Studio Code, you will need: - Administrator Privileges - Internet connectivity
Visual Studio Code License: https://code.visualstudio.com/license
Do you wish to proceed with the installation of Microsoft VSCode? [yes|no] >>> yes Proceeding with installation of Microsoft VSCode VSCode is already installed! ~/Software$
~/Software$ ls cuda_10.0.130_410.48_linux.run cudnn-10.0-linux-x64-v7.6.4.38.solitairetheme8 ~/Software$ chmod +x cuda_10.0.130_410.48_linux.run ~/Software$ ./cuda_10.0.130_410.48_linux.run Logging to /tmp/cuda_install_14812.log Using more to view the EULA. End User License Agreement --------------------------
Preface ------- The Software License Agreement in Chapter 1 and the Supplement in Chapter 2 contain license terms and conditions that govern the use of NVIDIA software. By accepting this agreement, you agree to comply with all the terms and conditions applicable to the product(s) included herein.
NVIDIA Driver Description This package contains the operating system driver and fundamental system software components for NVIDIA GPUs.
NVIDIA CUDA Toolkit Description
20. Licensee's use of linmath.h header for CPU functions for GL vector/matrix operations from lunarG is subject to the Apache License Version 2.0. ----------------- Do you accept the previously read EULA? accept/decline/quit: Do you accept the previously read EULA? accept/decline/quit: Do you accept the previously read EULA? accept/decline/quit: yes Do you accept the previously read EULA? accept/decline/quit: accept Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 410.48? (y)es/(n)o/(q)uit: no Install the CUDA 10.0 Toolkit? (y)es/(n)o/(q)uit: yes Enter Toolkit Location [ default is /usr/local/cuda-10.0 ]: /home/eustomaqua/Software/cuda-10.0 Do you want to install a symbolic link at /usr/local/cuda? (y)es/(n)o/(q)uit: no Install the CUDA 10.0 Samples? (y)es/(n)o/(q)uit: yes Enter CUDA Samples Location [ default is /home/eustomaqua ]: /home/eustomaqua/Software/cuda-samples Installing the CUDA Toolkit in /home/eustomaqua/Software/cuda-10.0 ... Missing recommended library: libGLU.so Missing recommended library: libXi.so Missing recommended library: libXmu.so Installing the CUDA Samples in /home/eustomaqua/Software/cuda-samples ... Copying samples to /home/eustomaqua/Software/cuda-samples/NVIDIA_CUDA-10.0_Samples now... Finished copying samples. =========== = Summary = =========== Driver: Not Selected Toolkit: Installed in /home/eustomaqua/Software/cuda-10.0 Samples: Installed in /home/eustomaqua/Software/cuda-samples, but missing recommended libraries Please make sure that - PATH includes /home/eustomaqua/Software/cuda-10.0/bin - LD_LIBRARY_PATH includes /home/eustomaqua/Software/cuda-10.0/lib64, or, add /home/eustomaqua/Software/cuda-10.0/lib64 to /etc/ld.so.conf and run ldconfig as root To uninstall the CUDA Toolkit, run the uninstall script in /home/eustomaqua/Software/cuda-10.0/bin Please see CUDA_Installation_Guide_Linux.pdf in /home/eustomaqua/Software/cuda-10.0/doc/pdf for detailed information on setting up CUDA. ***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 384.00 is required for CUDA 10.0 functionality to work. To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file: sudo <CudaInstaller>.run -silent -driver Logfile is /tmp/cuda_install_14812.log ~/Software$
\n\n
config .bashrc:
\n
1 2 3 4 5 6 7
~/Software$ vim ~/.bashrc ~/Software$ source ~/.bashrc ~/Software$ nvidia-smi Sat Feb 8 22:55:29 2020 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 440.36 Driver Version: 440.36 CUDA Version: 10.2 | |-------------------------------+----------------------+----------------------+
\n\n
end of .bashrc file:
\n
1 2 3 4 5
# added by Anaconda3 installer export PATH="/home/eustomaqua/anaconda3/bin:$PATH" # self added for Nvidia export PATH=$HOME/Software/cuda-10.0/bin:$PATH export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/Software/cuda-10.0/lib64/
~/Software$ ls cuda cudnn-10.0-linux-x64-v7.4.2.24.solitairetheme8 cuda-10.0 cudnn-10.0-linux-x64-v7.6.4.38.solitairetheme8 cuda_10.0.130_410.48_linux.run cudnn-10.0-linux-x64-v7.6.4.38.tgz cuda-samples ~/Software$ ~/Software$ ls cuda include lib64 NVIDIA_SLA_cuDNN_Support.txt ~/Software$ ls cuda/include cudnn.h ~/Software$ ls cuda/lib64 libcudnn.so libcudnn.so.7 libcudnn.so.7.6.4 libcudnn_static.a ~/Software$
~/Software$ cd .. ~$ ~$ python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import jupyter >>> import numpy as np >>> import pandas as pd >>> import sklearn >>> import matplotlib >>> from PIL import Image >>> import seaborn /home/eustomaqua/anaconda3/lib/python3.6/site-packages/matplotlib/font_manager.py:278: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. 'Matplotlib is building the font cache using fc-list. ' >>> >>> np.__version__ '1.14.3' >>> pd.__version__ '0.23.0' >>> seaborn.__version__ '0.8.1' >>> [4]+ Stopped python ~$ ~$ python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import seaborn >>> [5]+ Stopped python ~$
~$ pip config global.index-url # same as: pip config Need an action (edit, get, list, set, unset) to perform. You are using pip version 10.0.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip'command. ~$ pip config list global.index-url Got unexpected number of arguments, expected 0. (example: "pip config list") You are using pip version 10.0.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip'command. ~$ ~$ pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple Writing to /home/eustomaqua/.config/pip/pip.conf You are using pip version 10.0.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip'command.
~$ pip install opencv-python Requirement already satisfied: numpy>=1.11.3 in ./anaconda3/lib/python3.6/site-packages (from opencv-python) (1.14.3) distributed 1.21.8 requires msgpack, which is not installed. Installing collected packages: opencv-python Successfully installed opencv-python-4.2.0.32
~$ python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorflow as tf /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /home/eustomaqua/anaconda3/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) >>> [7]+ Stopped python
~$ pip install numpy==1.16.2 Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting numpy==1.16.2 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/35/d5/4f8410ac303e690144f0a0603c4b8fd3b986feb2749c435f7cdbb288f17e/numpy-1.16.2-cp36-cp36m-manylinux1_x86_64.whl (17.3MB) 100% |████████████████████████████████| 17.3MB 2.2MB/s Installing collected packages: numpy Found existing installation: numpy 1.18.1 Uninstalling numpy-1.18.1: Successfully uninstalled numpy-1.18.1 Successfully installed numpy-1.16.2 You are using pip version 10.0.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip'command.
~$ python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorflow as tf /home/eustomaqua/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters >>> [8]+ Stopped python
~$ pip install -U h5py Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting h5py Downloading https://pypi.tuna.tsinghua.edu.cn/packages/60/06/cafdd44889200e5438b897388f3075b52a8ef01f28a17366d91de0fa2d05/h5py-2.10.0-cp36-cp36m-manylinux1_x86_64.whl (2.9MB) 100% |████████████████████████████████| 2.9MB 13.5MB/s Requirement not upgraded as not directly required: numpy>=1.7 in ./anaconda3/lib/python3.6/site-packages (from h5py) (1.16.2) Requirement not upgraded as not directly required: six in ./anaconda3/lib/python3.6/site-packages (from h5py) (1.11.0) Installing collected packages: h5py Found existing installation: h5py 2.7.1 Uninstalling h5py-2.7.1: Successfully uninstalled h5py-2.7.1 Successfully installed h5py-2.10.0 You are using pip version 10.0.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip'command.
~$ python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorflow as tf >>> tf.test.is_gpu_available() 2020-02-08 23:52:36.691031: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-02-08 23:52:36.706948: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1 2020-02-08 23:52:36.853793: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-08 23:52:36.854165: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55f4173381e0 executing computations on platform CUDA. Devices: 2020-02-08 23:52:36.854183: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): GeForce RTX 2070 SUPER, Compute Capability 7.5 2020-02-08 23:52:36.873251: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3600000000 Hz 2020-02-08 23:52:36.873745: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55f417727de0 executing computations on platform Host. Devices: 2020-02-08 23:52:36.873759: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined> 2020-02-08 23:52:36.873898: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-08 23:52:36.874134: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: GeForce RTX 2070 SUPER major: 7 minor: 5 memoryClockRate(GHz): 1.8 pciBusID: 0000:01:00.0 2020-02-08 23:52:36.874223: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory 2020-02-08 23:52:36.874269: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory 2020-02-08 23:52:36.874312: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory 2020-02-08 23:52:36.874353: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory 2020-02-08 23:52:36.874395: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory 2020-02-08 23:52:36.874435: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory 2020-02-08 23:52:36.874476: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory 2020-02-08 23:52:36.874484: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices... 2020-02-08 23:52:36.874494: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-02-08 23:52:36.874499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2020-02-08 23:52:36.874504: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N False >>> [9]+ Stopped python
~$ LD_LIBRARY_PATH LD_LIBRARY_PATH: command not found ~$ source ~/.bashrc ~$ vim ~/.bashrc ~$ source ~/.bashrc ~$ LD_LIBRARY_PATH LD_LIBRARY_PATH: command not found ~$ ~$ ~$ ~$ vim ~/.bashrc ~$ source ~/.bashrc ~$ PATH PATH: command not found ~$ echo PATH PATH ~$ echo$PATH /home/eustomaqua/Software/cuda-10.0/bin:/home/eustomaqua/anaconda3/bin:/home/eustomaqua/Software/cuda-10.0/bin:/home/eustomaqua/anaconda3/bin:/home/eustomaqua/Software/cuda-10.0/bin:/home/eustomaqua/anaconda3/bin:/home/eustomaqua/Software/cuda-10.0/bin:/home/eustomaqua/anaconda3/bin:/home/eustomaqua/Software/cuda-10.0/bin:/home/eustomaqua/anaconda3/bin:/home/eustomaqua/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin ~$ echo$LD_LIBRARY_PATH :/home/eustomaqua/Software/cuda-10.0/lib64/:/home/eustomaqua/Software/cuda-10.0/lib64/
~$ python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license"for more information. >>> import tensorflow as tf >>> tf.test.is_gpu_available() 2020-02-09 00:08:17.326460: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-02-09 00:08:17.343444: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1 2020-02-09 00:08:17.381171: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-09 00:08:17.381486: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55d5564a5150 executing computations on platform CUDA. Devices: 2020-02-09 00:08:17.381501: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): GeForce RTX 2070 SUPER, Compute Capability 7.5 2020-02-09 00:08:17.405235: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3600000000 Hz 2020-02-09 00:08:17.405621: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55d556894ab0 executing computations on platform Host. Devices: 2020-02-09 00:08:17.405633: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined> 2020-02-09 00:08:17.405743: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-09 00:08:17.405958: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: GeForce RTX 2070 SUPER major: 7 minor: 5 memoryClockRate(GHz): 1.8 pciBusID: 0000:01:00.0 2020-02-09 00:08:17.406103: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2020-02-09 00:08:17.406736: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0 2020-02-09 00:08:17.407325: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0 2020-02-09 00:08:17.407470: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0 2020-02-09 00:08:17.408220: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0 2020-02-09 00:08:17.408804: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0 2020-02-09 00:08:17.410651: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2020-02-09 00:08:17.410718: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-09 00:08:17.410960: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-09 00:08:17.411148: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2020-02-09 00:08:17.411170: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2020-02-09 00:08:17.411687: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-02-09 00:08:17.411696: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2020-02-09 00:08:17.411700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2020-02-09 00:08:17.414113: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-09 00:08:17.414336: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-02-09 00:08:17.414546: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:0 with 7014 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2070 SUPER, pci bus id: 0000:01:00.0, compute capability: 7.5) True >>> [13]+ Stopped python
ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io$ hexo g Usage: hexo <command>
Commands: help Get help on a command. init Create a new Hexo folder. version Display version information.
Global Options: --config Specify config file instead of using _config.yml --cwd Specify the CWD --debug Display all verbose messages in the terminal --draft Display draft posts --safe Disable all plugins and scripts --silent Hide output on console
For more help, you can use 'hexo help [command]'for the detailed information or you can check the docs: http://hexo.io/docs/
ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io$ ls hexo-next README.md ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io$ cd hexo-next ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ hexo g # 生成 INFO Start processing INFO Files loaded in 3.29 s INFO Generated: search.xml INFO Generated: categories/index.html INFO Generated: .... INFO 114 files generated in 5.96 s ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ hexo s # 本地预览 INFO Start processing INFO Hexo is running at http://localhost:4000/. Press Ctrl+C to stop. ^CINFO Farewell ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ hexo d # 发布
ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ hexo g INFO Start processing INFO Files loaded in 5.74 s INFO Generated: 2020/2020-02-08-Setup-Python-on-Linux/index.html INFO Generated: search.xml INFO Generated: index.html INFO 3 files generated in 9.83 s ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$
ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io$ git add . ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io$ git commit -m "backup history" [hexo-next 2b17507] backup history 1 file changed, 51 insertions(+) ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io$ cd hex* ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ ls _config.yml debug.log package.json public source db.json node_modules package-lock.json scaffolds themes ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ hexo g INFO Start processing INFO Files loaded in 2.62 s INFO Generated: search.xml INFO Generated: 2020/2020-02-08-Setup-Python-on-Linux/index.html INFO Generated: index.html INFO 3 files generated in 6.39 s ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ hexo d INFO Deploying: git INFO Clearing .deploy_git folder... INFO Copying files from public folder... INFO Copying files from extend dirs... [master c64205a] updated on Sat, 02/08/2020 36 files changed, 3796 insertions(+), 79 deletions(-) create mode 100644 2020/2020-02-08-Setup-Python-on-Linux/index.html create mode 100644 archives/2020/02/index.html create mode 100644 archives/2020/index.html Username for'https://github.com': eustomaqua Password for'https://eustomaqua@github.com': To https://github.com/eustomaqua/eustomaqua.github.io.git df22d0c..c64205a HEAD -> master 分支 master 设置为跟踪来自 https://github.com/eustomaqua/eustomaqua.github.io.git 的远程分支 master。 INFO Deploy done: git ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$
\n\n
clean
\n
1 2 3 4
ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$ hexo clean INFO Deleted database. INFO Deleted public folder. ubuntu@ubuntu-VirtualBox:~/eustomaqua.github.io/hexo-next$
replace the device using a prepaid return label，但是他们无法寄到美国外的地址，失败
\n
update the software version (current 5.11.1) a. Home -> Setting -> Device Info 无更新选项，失败 b. 只能手动更新，下载固件，按网站提示进行 c. 更新完，原问题没解决
\n
word wise only works for English books (符合)
\n
power cycle the wifi modem (我换过三四个不同的 wifi，问题仍存在)
\n
try using a phone hotspot (翻不翻都一样结果，没用)
\n
最终结果是没解决
\n\n
后来在中亚测试一下 Word Wise 是否正常，中亚是正常的，而且十分快，我根本没看到 download 的提示 (不过我记得刚买到手的时候 Word Wise 的确有 download 提示，不过很快就下完了，印象中当时似乎没有下载词典，貌似没有主动去下载，不过词典不太记得了)。 于是再次换回美亚，仍是同样的问题。猜测难道服务器在国外，连接太慢所以下载不下来？
\n
再试一下下个词典试试 Word Wise，仍然是 “Downloading Word Wise…” 这个问题此前试过翻，无法解决…… 难道必须在当地才能用？主要是网速下载连接得上的问题？ 这也太惨了， 回国之后才发现幺蛾子都冒出来了，suddenlink 就坑了我一回，坑了我六十多刀都快三个月了还不还我，看来是不打算还了，这钱扔给 suddenlink 还不如续一年 student prime 呢，那个好歹还能看看 prime reading prime video [气愤ing]
$ mkdir software $ cd software # # install Python $ wget https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tgz $ tar -zxvf Python-3.6.1.tgz $ cd Python-3.6.1 $ ./configure --prefix=/home/userrname/software/python36 $ make -j $ make install # or: make -j && make install # 注意如果 make && make install 会失败，应该是权限问题 # # pip3 install package $ cd ../python36 $ bin/python3 $ bin/pip3 list $ cd bin $ ./pip3 install packagename
ubuntu@VirtualBox:~$ ssh -L16006:127.0.0.1:6006 yourname@server.address yourname@server.address's password: Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-141-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage 79 packages can be updated. 4 updates are security updates. New release '18.04.1 LTS' available. Run 'do-release-upgrade' to upgrade to it. *** System restart required *** Last login: Thu Feb 7 15:46:49 2019 from 10.231.238.49 yourname@server.address:~$ exit logout Connection to server.address closed. ubuntu@VirtualBox:~$
\n\n
使用
Usage: $ tensorboard --logdir=folder_path
\n\n
between Local Steps 1 & 2
\n
在服务器上开启 TensorBoard
1
tensorboard --logdir=./network
\n\n
e.g.,
\n
1 2 3 4 5 6 7 8 9 10
ubuntu@VirtualBox:~$ ssh -L16006:127.0.0.1:6006 yourname@server.address yourname@server.address's password: Last login: Wed Feb 27 11:08:00 2019 from 10.230.167.212 yourname@server.address:~$ source VirtualEnv/py35env/bin/activate (py35env) yourname@server.address:~$ cd yourfolder (py35env) yourname@server.address:~/yourfolder$ ls (py35env) yourname@server.address:~/yourfolder$ tensorboard --logdir=folder_of_your_model W0227 12:16:59.123035 Reloader tf_logging.py:120] Found more than one graph event per run, or there was a metagraph containing a graph_def, as well as one or more graph events. Overwriting the graph with the newest event. TensorBoard 1.12.1 at http://yourserver:6006 (Press CTRL+C to quit) ^C(py35env) yourname@server.address:~/yourfolder$
\n\n\n
Jupyter notebook 远程访问
仍以非 root 用户身份安装
\n
1, 安装步骤 (1) 登录服务器
\n
1 2 3
~$ source ~/VirtualEnv/py36env/bin/activate (py36env) ~$ jupyter notebook jupyter: command not found
(py36env) ~$ python Python 3.6.7 (default, Oct 212018, 04:56:05) [GCC 5.4.020160609] on linux Type "help", "copyright", "credits"or"license"for more information. >>> from IPython.lib import passwd >>> passwd() Enter password: Verify password: 'sha1:afc40e4de2b5:69f901707c23fde938709612d49424fab065c3b1' >>> [12]+ Stopped python
\n
(5) 修改默认配置文件
\n
1
(py36env) ~$ vim ~/.jupyter/jupyter_notebook_config.py
(py36env) ~$ jupyter notebook [I 12:16:27.663 NotebookApp] Serving notebooks from local directory: /home/username [I 12:16:27.663 NotebookApp] The Jupyter Notebook is running at: [I 12:16:27.664 NotebookApp] http://server.address:8888/ [I 12:16:27.664 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). ^C[I 12:23:05.617 NotebookApp] interrupted Serving notebooks from local directory: /home/username 0 active kernels The Jupyter Notebook is running at: http://username:8888/ Shutdown this notebook server (y/[n])? y [C 12:23:08.408 NotebookApp] Shutdown confirmed [I 12:23:08.409 NotebookApp] Shutting down 0 kernels (py36env) ~$
Traceback (most recent call last): File "/home/username/VirtualEnv/py36env/lib/python3.6/site-packages/notebook/notebookapp.py", line 864, in _default_allow_remote addr = ipaddress.ip_address(self.ip) File "/usr/lib/python3.6/ipaddress.py", line 54, in ip_address address) ValueError: 'username@server.address' does not appear to be an IPv4 or IPv6 address
\n
注意：c.NotebookApp.ip 的问题及报错
\n
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
#(1) c.NotebookApp.ip = 'username@server.address' ValueError: 'username@server.address' does not appear tobean IPv4 or IPv6 address
File -> Settings Project: your_name -> Project Interpreter add the python path on the server, remember to “Move this server to IDE settings“ instead of “Create copy of this deployment server in IDE settings” Edit Sync Folders (Local Path, Remote Path)
\n
其它配置
autopep8
File -> Settings -> Tools -> External Tools -> +
\n
\n
Program：C:\\Python35\\Scripts\\autopep8.exe Parameters：–in-place –ignore=E123,E133,E50 “$FilePath$“ Working directory：$FileDir$
\n\n\n\n","categories":["Records"],"tags":["Configuration","Linux","Git","Remote Server"]},{"title":"Logging in Python (日志处理)","url":"https://eustomaqua.github.io/2018/2018-09-18-Python-Logging/","content":"\n\n\n\n
## 配置 Logger # add formatter to ch ch.setFormatter(formatter) # add ch to logger logger.addHandler(ch) # The final log level is the higher one between the default and the one in handler
# 输出不同级别的log logger.debug('this is debug info') logger.info('this is information') logger.warn('this is warning message') logger.error('this is error message') logger.fatal('this is fatal message, it is same as logger.critical') logger.critical('this is critical message')
# 移除一些日志处理器 logger.removeHandler(file_handler)
\n\n
格式化输出日志
1 2 3 4 5 6 7 8 9
# 格式化输出
service_name = "Booking" logger.error('%s service is down!' % service_name) \t\t\t #使用Python自带的字符串格式化，不推荐 logger.error('%s service is down!', service_name) \t\t\t #使用logger的格式化，推荐 logger.error('%s service is %s!', service_name, 'down') \t #多参数格式化 logger.error('{} service is {}'.format(service_name, 'down')) #使用format函数，推荐
# 2018-09-18 21:11:34,493 ERROR : Booking service is down!
try: 1 / 0 except: # 等同于error级别，但是会额外记录当前抛出的异常堆栈信息 logger.exception('this is an exception message')
# 2016-10-08 21:59:19,493 ERROR : this is an exception message # Traceback (most recent call last): # File "D:/Git/py_labs/demo/use_logging.py", line 45, in # 1 / 0 # ZeroDivisionError: integer division or modulo by zero
\n","categories":["Coding"],"tags":["Commands","Python","Notes"]},{"title":"Git, Markdown, Linux, Vim 常用命令汇总","url":"https://eustomaqua.github.io/2018/2018-07-08-Git-Markdown-Linux-Vim/","content":"
cmp 命令用于比较两个文件是否有差异。 cut 命令用于显示每行从开头算起 num1 到 num2 的文字。 diff 命令用于比较文件的差异。逐行比较文本异同，如指定比较目录，则比较相同文件名的文件，但不比较其中的子目录。 diffstat 命令根据diff的比较结果，显示统计数字。 file 命令用于辨识文件类型。 find 命令用来在指定目录下查找文件。如不指定参数，则在当前目录下查找子目录与文件。 git 命令是文字模式下的文件管理员。 less 与 more 类似，但使用 less 可以随意浏览文件，而 more 仅能向前移动，却不能向后移动，而且 less 在查看之前不会加载整个文件。 more 命令类似 cat ，不过会以一页一页的形式显示，更方便使用者逐页阅读，而最基本的指令就是按空白键（space）就往下一页显示，按 b 键就会往回（back）一页显示，而且还有搜寻字串的功能（与 vi 相似），使用中的说明文件，请按 h 。 od 命令用于输出文件内容。它读取所给予的文件的内容，并将其内容以八进制字码呈现出来。 paste 命令用于合并文件的列。 patch 命令用于修补文件。 touch 命令用于修改文件或者目录的时间属性，包括存取时间和更改时间。若文件不存在，系统会建立一个新的文件。 ls -l 可以显示档案的时间记录。 slocate 命令查找文件或目录，本身有一个数据库，里面存放了系统中文件与目录的相关信息。 split 命令用于将大文件分割成较小的文件，在默认情况下按照每 1000 行切割。 tee 命令用于读取标准输入的数据，并将内容输出到标准输出设备，同时保存成文件。 read 命令用于从标准输入读取数值。
date 命令可以用来显示或设定系统的日期与时间。 id 命令用于显示用户的 ID，以及所属群组的 ID。 who 命令用于显示系统中有哪些使用者正在上面，显示的资料包含了使用者 ID、使用的终端机、从哪边连上来的、上线时间、呆滞时间、CPU 使用量、动作等等。 logout 命令用于退出系统。 free 命令用于显示内存状态。会显示内存的使用情况，包括实体内存，虚拟的交换文件内存，共享内存区段，以及系统核心使用的缓冲区等。
![](\"/images/2024-12/20190407161507623.png\")
\n![](\"/images/2021-01/0105_CUDA10.0_reinstall.png\")
\n\n![](\"/images/2024-12/20200307194357123.png\")
\n之后可以清理下注册表，保险起见可以重启，但好像不重启也挺好？\n\n![](\"/images/2021-01/0105_NVIDIA_control.png\")
\n这个代表电脑显卡能支持的 cuda 最大版本 \n![](\"/images/2024-12/2019040715251354.png\")
\n特别是这个 Visual Studio Integration 千万不能选！\n\n![](\"/images/2024-12/2019040715270168.png\")
\n选择以下的安装就够了\n\n![](\"/images/2024-12/20190407152642126.png\")
\n安装到 C 盘即可，方便以后的路径等\n\n![](\"/images/2024-12/20190407152738412.png\")
![](\"/images/2024-12/20190407152806986.png\")
\n![](\"/images/2024-12/2019040715282075.png\")
![](\"/images/2024-12/20200409104116584.png\")
\n![](\"/images/2024-12/20200409104130470.png\")
![](\"https://pic2.zhimg.com/80/v2-262e209917537c0f699cc1a5eb803b2d_720w.png\")
![\"avatar\"](\"/images/2018-09/logging_flow.png\")
![](\"/images/2018-07/09_git_commit_a.png\")
![\"git\"](\"http://www.ruanyifeng.com/blogimg/asset/2015/bg2015120901.png\")
![\"Vim/Vi](\"http://www.runoob.com/wp-content/uploads/2014/07/vim-vi-workmodel.png\")
![\"vim](\"http://www.runoob.com/wp-content/uploads/2015/10/vi-vim-cheat-sheet-sch.gif\")
![](\"/images/2018-07/05_02_lmdb_img1.png\")
![](\"/images/2018-07/05_02_lmdb_img2.png\")
![](\"/images/2024-12/crud-annotated-document.png\")
\n\n![](\"/images/2018-07/07_mongodb_img1.png\")
\n![](\"/images/2018-07/07_mongodb_img2.png\")
\n\n![](\"/images/2018-07/07_robo3t_img1.png\")
\n![](\"/images/2018-07/07_robo3t_img2.png\")
\n![](\"/images/2018-07/07_robo3t_img3.png\")
\n\n![](\"/images/2018-07/07_use_db_img1.png\")
\n![](\"/images/2018-07/07_use_db_img5.png\")
\n\n![](\"/images/2018-07/07_sublime_img1.png\")
![](\"/images/2018-07/07_sublime_img2.png\")
![](\"/images/2018-07/07_sublime_img3.png\")
![](\"/images/2018-07/07_sublime_img4.png\")
![](\"/images/2018-07/07_weibo_img1.png\")
\n![](\"/images/2018-07/07_weibo_img2.png\")
\n![](\"/images/2018-07/07_weibo_img3.png\")
\n\n\n\n