Add size-gated hugepage optimization for large allocations#291
Add size-gated hugepage optimization for large allocations#291hsane-dev wants to merge 1 commit intogoogle:masterfrom
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
d24787f to
f0bf91a
Compare
|
I could imagine this still causes regressions for some people -- eg if you aren't expecting hugepages and use an arena allocator with a 2MiB allocation size, now you'll get a full 2M RSS whereas before you got granular 4k page allocation as your arena filled up? |
|
I think I'm comfortable with someone getting a hugepage: TCMalloc likes hugepages, and manages memory in a hugepage-aware way. We're assuming they're there, even if they aren't, when making allocation decisions. That said, I'm not sure the size restriction buys us much. Allocations to the system allocator are typically making 1GB-sized allocations, and we end up faulting in the memory later when it |
|
MADV_HUGEPAGE is advisory only (no prefault; RSS grows on touch) and only has effect when THP is set to madvise. @ckennelly agree the >=2MiB gate likely doesn’t buy much if typical 1GiB system allocations—happy to drop the threshold |
- Add madvise_hugepage_mmap_enabled parameter (default false) - Remove size gating per @ckennelly's feedback (TCMalloc operates in hugepage chunks) - Advisory-only madvise calls with error restoration - Runtime configurable via TCMalloc_Internal_SetMadviseHugepageMmapEnabled() - No behavior change unless explicitly enabled The optimization opts into transparent hugepages when the system is configured for transparent_hugepage=madvise. This can improve memory performance by reducing TLB pressure for large allocations. Follows TCMalloc parameter patterns with atomic storage and weak function accessors for runtime configuration.
f0bf91a to
78cb21e
Compare
|
Summary
Adds proactive hugepage optimization to improve memory performance for allocations ≥2MB while preserving existing memory management behavior.
Changes
MmapRegion::Alloc()insystem_allocator.hto applyMADV_HUGEPAGEfor large allocations>= kHugePageSize) to avoid overhead on small allocationsMADV_NOHUGEPAGElogic for infrequent access patterns