improve memory cleanup of job generator msg processor

[mightqxc]

No description provided.

[Copilot]

Pull request overview

Improves the JediJobGeneratorMsgProcPlugin message processor’s memory behavior by adding short-lived caches for heavy metadata, reusing TaskSetupper instances, and triggering explicit cleanup/heap trimming when memory usage is high.

Changes:

• Add TTL-based caching for site/work-queue/resource-type metadata and per-queue nFiles config.
• Reuse TaskSetupper instances across messages to reduce repeated initialization.
• Add a finally cleanup block and conditional gc.collect() + malloc_trim when RSS exceeds a threshold.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.