Python with packages depending on Numpy – part II

Solution 2 – Fork the Buildpack

This next solution will require some Go-Language programming skills.

I’ve create a fork on the python buildpack. If you are interested on testing my solution, there is no need to create a fork yourself. All you need to do is to change your manifest to point to my buildpack:

---
applications:
- name: flask-skfuzzy
  memory: 512MB
  disk_quota: 2G
  buildpacks:
    - https://github.com/ivanmir/python-buildpack#numpy
  #https://github.com/cloudfoundry/python-buildpack
  stack: cflinuxfs3
  env:
    #BP_DEBUG: "False"

NOTE: Notice that I have created a new branch called “numpy” on my forked buildpack. This is maintained in the manifest by including the ‘#’ with the branch/tag name.

Once you do this, all is needed is to have restore the requirements.txt and the runtime.txt as it will pick things from there.

Keep in mind that this solution will install numpy, matplotlib and scipy. Thus, there is no need to include them on your requirements.txt file. Mine looks like this:

Flask==1.0.2
scikit-fuzzy==0.4.0

NOTE: This simplifies the CF installation a lot and the time it takes to push your app. However, I didn’t take the time to install process the requirements.txt to read version numbers, etc. Therefore, it will install the latest versions compatible with the python runtime you specified on runtime.txt file.

python-3.6.8

When you push the application to CF you will notice a new line. That’s my contribution to this forked buildpack:

Stopping app...

Staging app and tracing logs...
   Cell 5a42b8a2-...-b047ba71c478 creating container for instance 50fc246d-...-f0280b6add87
   Cell 5a42b8a2-...-b047ba71c478 successfully created container for instance 50fc246d-...-f0280b6add87
   Downloading app package...
   Downloading build artifacts cache...
   Downloaded app package (6.8K)
   Downloaded build artifacts cache (242.3M)
   -----> Download go 1.11.4
   -----> Running go build supply
   /tmp/buildpackdownloads/93f606e89f788e3d80357a91d4890d0e ~
   ~
   -----> Python Buildpack version 1.6.28
   -----> Supplying Python
   -----> Installing python 3.6.8
          Download [https://buildpacks.cloudfoundry.org/dependencies/python/python-3.6.8-linux-x64-cflinuxfs3-0e8b91a8.tgz]
   -----> Installing pip-pop 0.1.3
          Download [https://buildpacks.cloudfoundry.org/dependencies/manual-binaries/pip-pop/pip-pop-0.1.3-fc106ef6.tar.gz]
          Installing ML libs
          ML libs installed.

The good part of this approach is that the resource usage is way less (memory and disk) than what I’ve achieved with miniconda on my previous blog.

Statistic while using the miniconda approach:

name:              flask-skfuzzy
requested state:   started
routes:            flask-skfuzzy.cfapps.eu10.hana.ondemand.com
last uploaded:     Fri 15 Feb 14:30:53 DST 2019
stack:             cflinuxfs3
buildpacks:        python

type:            web
instances:       1/1
memory usage:    512M
start command:   python app.py
     state     since                  cpu    memory          disk       details
#0   running   2019-02-15T16:31:43Z   1.2%   98.6M of 512M   1G of 2G

Now with the forked buildpack:

name:              flask-skfuzzy
requested state:   started
routes:            flask-skfuzzy.cfapps.eu10.hana.ondemand.com
last uploaded:     Fri 15 Feb 16:22:42 DST 2019
stack:             cflinuxfs3
buildpacks:        python

type:            web
instances:       1/1
memory usage:    512M
start command:   python app.py
     state     since                  cpu    memory        disk         details
#0   running   2019-02-15T18:23:12Z   0.0%   40K of 512M   366M of 2G

Enjoy!