~johanvandegriff/tensorflow-custom-build

89b2901ae9dca363b3e0a30148b23efc4cbf2e3e — Johan Vandegriff 10 months ago
initial files
2 files changed, 87 insertions(+), 0 deletions(-)

A README.md
A tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl
A  => README.md +87 -0
@@ 1,87 @@
# The Issue
I was getting `Illegal instruction (core dumped)` when running tensorflow 2.4.1 on my computer. I found [this github issue](https://github.com/tensorflow/tensorflow/issues/17411) which said that newer versions of tensorflow require CPUs with the AVX instruction, which mine apparently does not have. The solution suggested in that thread was to build from source, which solved it for me, but took several hours to complete. So I decided to publish the build for anyone who has the same issue and trusts me to not modify the build. If you want better security, build it yourself using the process in [this guide](https://www.tensorflow.org/install/source) or the rest of this document.

# My Build
Download it here:
[tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl](tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl)

And install it with
```
pip3 install tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl
```

This was built with the flags `-march=native -mcx16` for 64-bit linux on the following CPU:
```
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       40 bits physical, 48 bits virtual
CPU(s):              16
On-line CPU(s) list: 0-15
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               26
Model name:          Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
Stepping:            5
CPU MHz:             1646.208
CPU max MHz:         2267.0000
CPU min MHz:         1600.0000
BogoMIPS:            4533.42
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            8192K
NUMA node0 CPU(s):   0-3,8-11
NUMA node1 CPU(s):   4-7,12-15
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm pti tpr_shadow vnmi flexpriority ept vpid dtherm
```


# Build Process
The following commands are what I ran based on [this document](#https://www.tensorflow.org/install/source). I also built this inside a docker container for my own convenience, but that is not needed. Note that you have to paste in the flags that are outputted from the first step into the `./configure` when it asks you.
```
#find compiler flags to use later in ./configure
grep flags -m1 /proc/cpuinfo | cut -d ":" -f 2 | tr '[:upper:]' '[:lower:]' | { read FLAGS; OPT="-march=native"; for flag in $FLAGS; do case "$flag" in "sse4_1" | "sse4_2" | "ssse3" | "fma" | "cx16" | "popcnt" | "avx" | "avx2") OPT+=" -m$flag";; esac; done; MODOPT=${OPT//_/\.}; echo "$MODOPT"; }
#-march=native -mcx16

#apt dependencies
apt update
apt install python3-dev python3-pip -y

#pip dependencies
pip3 install -U --user pip numpy wheel
pip3 install -U --user keras_preprocessing --no-deps

#install bazel
wget https://github.com/bazelbuild/bazel/releases/download/3.7.2/bazel-3.7.2-linux-x86_64
chmod +x bazel-3.7.2-linux-x86_64
cp bazel-3.7.2-linux-x86_64 /bin/bazel

## SKIP THIS SECTION ##
#this version of bazel was too new:
#wget https://github.com/bazelbuild/bazelisk/releases/download/v1.7.4/bazelisk-linux-amd64
#chmod +x bazelisk-linux-amd64
#./bazelisk-linux-amd64
#cp /root/.cache/bazelisk/downloads/bazelbuild/bazel-4.0.0-linux-x86_64/bin/bazel /bin/bazel
## END OF SECTION TO SKIP ##

git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow

git checkout r2.4
./configure
#IMPORTANT!!! enter the compiler flags when prompted "Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -Wno-sign-compare]: "

#build the actual thing -- takes a while!
bazel build //tensorflow/tools/pip_package:build_pip_package

#build the pip package, takes only a few minutes
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
```

This will drop a file in `/tmp/tensorflow_pkg`, for me this was called `tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl` but it might be different for you, depending on which version you selected with `git checkout r2.4`. Save this file somewhere so you don't have to rebuild to get it again. You can install the file with pip, and tensorflow will work on your CPU without AVX support.

A  => tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl +0 -0