Track your Python memory Allocations - SAP BAS

[yogananda]

Dear All,

This article is intended for developers who are working on python development in SAP Business Application Studio can test your code for memory profile.

What is are Python memory profilers?

Profiling applications always involve issues such as CPU, memory, etc. However, Python applications are prone to memory management issues. This is primarily because Python is applied to Data Science and ML applications and works with vast amounts of data. Also, Python relies on its Memory Management system by default, instead of leaving it to the user.

As Python code works within containers via a distributed processing framework, each container contains a fixed amount of memory. If the code execution exceeds the memory limit, then the container will terminate. This is when development experiences memory errors

---

Memory profiling is a very powerful technique that makes it possible to understand how a program uses memory, thus helping to detect memory leaks or understand which parts of the program cause the most memory usage.

Developed at Bloomberg and now open source, Memray is a memory profiler that can track memory allocations in Python code, including native extensions and in the Python interpreter itself.

Memray is able to generate various kinds of reports about the collected memory usage data, including flame graphs, which are useful to identify the most frequent code-paths quickly and accurately.

The tool is a must for any long-running services implemented with Python. With memray you can generate flame charts or all allocations and trace absolutely everything.



Memray can be used as a command-line tool to run and profile a Python program:

Instructions :

https://github.com/bloomberg/memray

 python3 -m pip install memray

---

Step 1 : Install Python & Pip3 in Business Application Studio ( Ignore if its already installed)

Step 2 : Clone the sample python from my git

https://github.com/yogananda-muthaiah/memray_sample.git

Step 3 : Test it once in your local SAP Business Application Studio

python3 app.py

Step 4 : run the memray which will throw a bin file in your folder

python3 -m memray run app.py





Writing profile results into memray-app.py.1580.bin

emray WARNING: Correcting symbol for malloc from 0x4215f0 to 0x7f98465713e0

Memray WARNING: Correcting symbol for free from 0x421a50 to 0x7f9846571a30

Memray WARNING: Correcting symbol for aligned_alloc from 0x7f984600b480 to 0x7f98465720f0

[########################################]

[memray] Successfully generated profile results.



You can now generate reports from the stored allocation records.

Some example commands to generate reports:



/usr/bin/python3 -m memray flamegraph memray-app.py.1580.bin

Step 5 : run the flamegraph using memray with downloaded bin file

python3 -m memray flamegraph memray-app.py.1580.bin



Wrote memray-flamegraph-app.py.1580.html

Step 6 : If you need live memory profile - run the below command

python3 -m memray run --live app.py