To profile memory usage of your Python code, you can use the memory_profiler package. This package provides a decorator @profile that you can use to profile the memory usage of individual functions.
Here is an example of how to use memory_profiler:
!pip install memory_profilerimport randomfrom memory_profiler import profile@profiledef my_function(): nums = [random.randint(0, 100) for _ in range(1000000)] sorted_nums = sorted(nums) return sorted_numsmy_function() |
In this example, the @profile decorator is used to profile the memory usage of the my_function function. When the function is executed, memory_profiler will print a detailed report of the memory usage of each line of code in the function.
Here is an example of what the output might look like:
Line # Mem usage Increment Line Contents================================================ 4 54.4 MiB 54.4 MiB @profile 5 def my_function(): 6 132.7 MiB 0.0 MiB nums = [random.randint(0, 100) for _ in range(1000000)] 7 323.3 MiB 190.6 MiB sorted_nums = sorted(nums) 8 323.3 MiB 0.0 MiB return sorted_nums |
The output shows the memory usage of each line of code in the function, as well as the increment in memory usage between each line. In this example, you can see that most of the memory usage is due to the nums list at line 6, which takes up 132.7 MiB. The sorted_nums list at line 7 takes up an additional 190.6 MiB of memory.
You can use the information from the memory_profiler output to optimize your code and reduce memory usage. For example, you might consider using a generator instead of a list to avoid loading all the data into memory at once.