Python with packages depending on Numpy – part II
Solution 2 – Fork the Buildpack
This next solution will require some Go-Language programming skills.
I’ve create a fork on the python buildpack. If you are interested on testing my solution, there is no need to create a fork yourself. All you need to do is to change your manifest to point to my buildpack:
--- applications: - name: flask-skfuzzy memory: 512MB disk_quota: 2G buildpacks: - https://github.com/ivanmir/python-buildpack#numpy #https://github.com/cloudfoundry/python-buildpack stack: cflinuxfs3 env: #BP_DEBUG: "False"
NOTE: Notice that I have created a new branch called “numpy” on my forked buildpack. This is maintained in the manifest by including the ‘#’ with the branch/tag name.
Once you do this, all is needed is to have restore the requirements.txt and the runtime.txt as it will pick things from there.
Keep in mind that this solution will install numpy, matplotlib and scipy. Thus, there is no need to include them on your requirements.txt file. Mine looks like this:
NOTE: This simplifies the CF installation a lot and the time it takes to push your app. However, I didn’t take the time to install process the requirements.txt to read version numbers, etc. Therefore, it will install the latest versions compatible with the python runtime you specified on runtime.txt file.
When you push the application to CF you will notice a new line. That’s my contribution to this forked buildpack:
Stopping app... Staging app and tracing logs... Cell 5a42b8a2-...-b047ba71c478 creating container for instance 50fc246d-...-f0280b6add87 Cell 5a42b8a2-...-b047ba71c478 successfully created container for instance 50fc246d-...-f0280b6add87 Downloading app package... Downloading build artifacts cache... Downloaded app package (6.8K) Downloaded build artifacts cache (242.3M) -----> Download go 1.11.4 -----> Running go build supply /tmp/buildpackdownloads/93f606e89f788e3d80357a91d4890d0e ~ ~ -----> Python Buildpack version 1.6.28 -----> Supplying Python -----> Installing python 3.6.8 Download [https://buildpacks.cloudfoundry.org/dependencies/python/python-3.6.8-linux-x64-cflinuxfs3-0e8b91a8.tgz] -----> Installing pip-pop 0.1.3 Download [https://buildpacks.cloudfoundry.org/dependencies/manual-binaries/pip-pop/pip-pop-0.1.3-fc106ef6.tar.gz] Installing ML libs ML libs installed.
The good part of this approach is that the resource usage is way less (memory and disk) than what I’ve achieved with miniconda on my previous blog.
Statistic while using the miniconda approach:
name: flask-skfuzzy requested state: started routes: flask-skfuzzy.cfapps.eu10.hana.ondemand.com last uploaded: Fri 15 Feb 14:30:53 DST 2019 stack: cflinuxfs3 buildpacks: python type: web instances: 1/1 memory usage: 512M start command: python app.py state since cpu memory disk details #0 running 2019-02-15T16:31:43Z 1.2% 98.6M of 512M 1G of 2G
Now with the forked buildpack:
name: flask-skfuzzy requested state: started routes: flask-skfuzzy.cfapps.eu10.hana.ondemand.com last uploaded: Fri 15 Feb 16:22:42 DST 2019 stack: cflinuxfs3 buildpacks: python type: web instances: 1/1 memory usage: 512M start command: python app.py state since cpu memory disk details #0 running 2019-02-15T18:23:12Z 0.0% 40K of 512M 366M of 2G