#983 closed defect (fixed)
maintenance/cleanupWebSessions.py does not scale
| Reported by: | makub | Owned by: | jatrzask |
|---|---|---|---|
| Priority: | critical | Milestone: | v0.99.0 |
| Component: | General | Version: | 0.97.0 |
| Keywords: | Cc: |
Description
The script maintenance/cleanupWebSessions.py does not work for millions of web sessions, because it first collects all sessions keys, and then removes them. The script exhausts memory on a machine with 4GB memory.
The solution is to simply find and remove sessions in batches of small sizes, like 1000.
Attachments (3)
Change History (10)
Changed 4 years ago by makub
comment:1 Changed 4 years ago by jbenito
- Milestone set to v0.98.1
comment:2 Changed 3 years ago by jbenito
- Milestone changed from v0.98.1 to v0.99.0
- Owner set to jatrzask
- Priority changed from normal to critical
- Status changed from new to assigned
comment:3 Changed 3 years ago by jatrzask
- Status changed from assigned to in_work
comment:4 Changed 3 years ago by jatrzask
- Status changed from in_work to awaiting_merge
comment:5 Changed 3 years ago by pferreir
- Status changed from awaiting_merge to merging
Martin,
Thanks for the contribution, but I believe there is a better solution which involves using iterators.
I will add the fix, and take the opportunity to finally move this to a scheduler task.
Give it a try and let us know, please.
comment:6 Changed 3 years ago by Jakub Trzaskoma <jakub.piotr.trzaskoma@…>
- Resolution set to fixed
- Status changed from merging to closed
comment:7 Changed 3 years ago by pferreir
fe5e148046b7acba3cb92ea1f09a107a3f622fc9 contains the solution using iterators (see janitor.py).
Note: See
TracTickets for help on using
tickets.
a script that deletes in batches