Skip to content

Reduced chunk_size to reduce memory fragmentation #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 3, 2020

Conversation

cogliano
Copy link
Contributor

@cogliano cogliano commented Feb 2, 2020

Running a PyPortal Titano on CP 5.0.0-beta.4, I was getting MemoryError exceptions after 10 to 20 fetch() calls within a loop. Further research showed the memory was not low, but apparently fragmented. I added some gc.collect() calls in different functions, but that did not help. Finally, I reduced the chunk_size value from 12000 to 4096. This helped greatly, and I was able to get over 300 fetch loops (and counting) without the exception. I suggest implementing this solution until a better solution is found to address memory fragmentation with a higher higher chunk_size value.

The code used to test this issue is here:

https://github.com/adafruit/Adafruit_Learning_System_Guides/tree/master/PyPortal_CMA_Art_Frame

You can reduce the while loop wait to 60 seconds to help speed up the looping for testing.

@ladyada
Copy link
Member

ladyada commented Feb 2, 2020

did image download time increase significantly?

@cogliano
Copy link
Contributor Author

cogliano commented Feb 3, 2020

Surprisingly, the download time decreased by about 2 seconds, perhaps due to less garbage collection in the loop? Here are my results using the Cleveland Museum learning guide code downloading the same image 3 times with the different chunk sizes:

chunk_size = 12000:

Created file of 296082 bytes in 12.6 seconds
Created file of 296082 bytes in 12.7 seconds
Created file of 296082 bytes in 12.7 seconds

chunk_size = 4096:

Created file of 296082 bytes in 10.6 seconds
Created file of 296082 bytes in 10.4 seconds
Created file of 296082 bytes in 10.5 seconds

@ladyada ladyada merged commit ad5cabf into adafruit:master Feb 3, 2020
adafruit-adabot added a commit to adafruit/Adafruit_CircuitPython_Bundle that referenced this pull request Feb 18, 2020
Updating https://github.com/adafruit/Adafruit_CircuitPython_ADS1x15 to 2.1.3 from 2.1.2:
  > Merge pull request adafruit/Adafruit_CircuitPython_ADS1x15#52 from ares-est/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_CLUE to 2.0.3 from 2.0.1:
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#17 from kattni/slideshow-fix
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#14 from dherrada/master
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#16 from kattni/slideshow-example
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#15 from kattni/add-display-object
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#12 from kattni/add-space
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#11 from kattni/variable-change
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#10 from kattni/level-bubble-fix
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#9 from kattni/color-fix
  > Merge pull request adafruit/Adafruit_CircuitPython_CLUE#8 from kattni/example-update

Updating https://github.com/adafruit/Adafruit_CircuitPython_Fingerprint to 1.1.6 from 1.1.5:
  > Merge pull request adafruit/Adafruit_CircuitPython_Fingerprint#11 from stitchesnburns/stitchesnburns-patch-1

Updating https://github.com/adafruit/Adafruit_CircuitPython_LIS2MDL to 2.0.2 from 2.0.1:
  > Update README.rst

Updating https://github.com/adafruit/Adafruit_CircuitPython_LSM303_Accel to 1.0.3 from 1.0.2:
  > Merge pull request adafruit/Adafruit_CircuitPython_LSM303_Accel#5 from BiffoBear/rename-example-file
  > Merge pull request adafruit/Adafruit_CircuitPython_LSM303_Accel#4 from FoamyGuy/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_LSM303DLH_Mag to 1.0.4 from 1.0.3:
  > Merge pull request adafruit/Adafruit_CircuitPython_LSM303DLH_Mag#5 from FoamyGuy/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_MCP230xx to 2.2.3 from 2.2.2:
  > Merge pull request adafruit/Adafruit_CircuitPython_MCP230xx#21 from foozmeat/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_MPU6050 to 1.0.4 from 1.0.3:
  > Merge pull request adafruit/Adafruit_CircuitPython_MPU6050#5 from FoamyGuy/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_PyPortal to 3.1.10 from 3.1.9:
  > Merge pull request adafruit/Adafruit_CircuitPython_PyPortal#65 from cogliano/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_RGB_Display to 3.8.9 from 3.8.8:
  > Merge pull request adafruit/Adafruit_CircuitPython_RGB_Display#70 from makermelissa/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_BluefruitConnect to 1.0.11 from 1.0.10:
  > Merge pull request adafruit/Adafruit_CircuitPython_BluefruitConnect#17 from caternuson/controller_example

Updating https://github.com/adafruit/Adafruit_CircuitPython_BusDevice to 4.2.0 from 4.1.4:
  > Merge pull request adafruit/Adafruit_CircuitPython_BusDevice#43 from dhalbert/avoid-alloc

Updating https://github.com/adafruit/Adafruit_CircuitPython_Gizmo to 1.1.3 from 1.1.2:
  > Merge pull request adafruit/Adafruit_CircuitPython_Gizmo#10 from FoamyGuy/master

Updating https://github.com/adafruit/Adafruit_CircuitPython_Bundle/circuitpython_library_list.md to NA from NA:
  > Added the following libraries: Adafruit_CircuitPython_LPS2X
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants