ubuntu - How to enable multithreading with Caffe? -
i compile / configure caffe when trained artificial neural network it, training multi-threaded (cpu only, no gpu). how enable multithreading caffe? use caffe on ubuntu 14.04 lts x64.
one way use openblas instead of default atlas. so,
sudo apt-get install -y libopenblas-dev
- before compiling caffe, edit
makefile.config
, replaceblas := atlas
blas := open
- after compiling caffe, running
export openblas_num_threads=4
cause caffe use 4 cores.
if interested, here script install caffe , pycaffe on new ubuntu 14.04 lts x64 or ubuntu 14.10 x64. cpu only, multi-threaded caffe. can improved, it's enough me now:
# script installs caffe , pycaffe on ubuntu 14.04 x64 or 14.10 x64. cpu only, multi-threaded caffe. # usage: # 0. set here how many cores want use during installation: # default caffe use these cores. number_of_cores=4 # 1. execute script, e.g. "bash compile_caffe_ubuntu_14.04.sh" (~30 60 minutes on new ubuntu). # 2. open new shell (or run "source ~/.bash_profile"). you're done. can try # running "import caffe" python interpreter test. #http://caffe.berkeleyvision.org/install_apt.html : (general install info: http://caffe.berkeleyvision.org/installation.html) cd sudo apt-get update #sudo apt-get upgrade -y # if ok getting prompted sudo debian_frontend=noninteractive apt-get upgrade -y -q -o dpkg::options::="--force-confdef" -o dpkg::options::="--force-confold" # if ok defaults sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev sudo apt-get install -y --no-install-recommends libboost-all-dev sudo apt-get install -y libatlas-base-dev sudo apt-get install -y python-dev sudo apt-get install -y python-pip git # ubuntu 14.04 sudo apt-get install -y libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler # lmdb # https://github.com/bvlc/caffe/issues/2729: temporarily broken link lmdb repository #2729 #git clone https://gitorious.org/mdb/mdb.git #cd mdb/libraries/liblmdb #make && make install git clone https://github.com/lmdb/lmdb.git cd lmdb/libraries/liblmdb sudo make sudo make install # more pre-requisites sudo apt-get install -y cmake unzip doxygen sudo apt-get install -y protobuf-compiler sudo apt-get install -y libffi-dev python-dev build-essential sudo pip install lmdb sudo pip install numpy sudo apt-get install -y python-numpy sudo apt-get install -y gfortran # required scipy sudo pip install scipy # required scikit-image sudo apt-get install -y python-scipy # in case pip failed sudo apt-get install -y python-nose sudo pip install scikit-image # fix https://github.com/bvlc/caffe/issues/50 # caffe (http://caffe.berkeleyvision.org/installation.html#compilation) cd mkdir caffe cd caffe wget https://github.com/bvlc/caffe/archive/master.zip unzip -o master.zip cd caffe-master # prepare python binding (pycaffe) cd python req in $(cat requirements.txt); sudo pip install $req; done echo "export pythonpath=$(pwd):$pythonpath " >> ~/.bash_profile # able call "import caffe" python after reboot source ~/.bash_profile # update shell cd .. # compile caffe , pycaffe cp makefile.config.example makefile.config sed -i '8s/.*/cpu_only := 1/' makefile.config # line 8: cpu sudo apt-get install -y libopenblas-dev sed -i '33s/.*/blas := open/' makefile.config # line 33: use openblas # note if 1 day makefile.config changes , these line numbers change, we're screwed # maybe best append changes @ end of makefile.config echo "export openblas_num_threads=($number_of_cores)" >> ~/.bash_profile mkdir build cd build cmake .. cd .. make -j$number_of_cores # 4 number of parallel threads compilation: typically equal number of physical cores make pycaffe -j$number_of_cores make test make runtest #make matcaffe make distribute # bonus other work pycaffe sudo pip install pydot sudo apt-get install -y graphviz sudo pip install scikit-learn # @ end, need run "source ~/.bash_profile" manually or start new shell able 'python import caffe', # because 1 cannot source in bash script. (http://stackoverflow.com/questions/16011245/source-files-in-a-bash-script)
i have placed script on github:
https://github.com/franck-dernoncourt/caffe_demos/tree/master/caffe_installation .
Comments
Post a Comment