Optimized string matching for RoleBasedActionPrivileges

[nibix]

Description

Fixes #5998

This PR provides two optimizations:

• All compilation of index patterns, static DLS queries, etc. is decoupled from building RoleBasedActionPrivileges and AbstractRuleBasedPrivileges into a new class CompiledRoles. This will guarantee that each index pattern and static DLS query is only compiled once even if it is later used in different components (e.g. stateful vs non-stateful classes or index privileges vs DLS) and for different indices.
• The IndexPattern class is extended to provide optimized matching for prefix type index patterns (like index_*) and for exact index patterns (like index_a). For prefix type index patterns, we leverage the fact that the cluster state contains a TreeSet of all indices, aliases and data streams. Performing a prefix pattern match is then only a O(log n) operation, while it was an O(n) operation before.

These optimizations reduces the runtime of stateful index privileges in one micro benchmark test case by a factor of about 0.46.

Before:

[constructionPerformance_sharedPatterns] Configuration: 3000 roles, 6000 indices, 9000 aliases
[constructionPerformance_sharedPatterns] RoleBasedActionPrivileges construction: 148ms
[constructionPerformance_sharedPatterns] StatefulIndexPrivileges update: 2169ms
[constructionPerformance_sharedPatterns] Total: 2317ms

After:

[constructionPerformance_sharedPatterns] Configuration: 3000 roles, 6000 indices, 9000 aliases
[constructionPerformance_sharedPatterns] RoleBasedActionPrivileges construction: 42ms
[constructionPerformance_sharedPatterns] StatefulIndexPrivileges update: 971ms
[constructionPerformance_sharedPatterns] Total: 1013ms

• Category
  • Enhancement
• Why these changes are required?
  • In clusters with many roles and indices, the update of role configuration could have a latency of several seconds. This will now the reduce the latency by about 50%.
• What is the old behavior before changes and new behavior after changes?
  • No behavioral changes.

Testing

• Existing integration tests

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

❌ Patch coverage is 90.68826% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.85%. Comparing base (d9b4ccc) to head (8d183e2).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../opensearch/security/privileges/CompiledRoles.java	78.57%	10 Missing and 2 partials ⚠️
...g/opensearch/security/privileges/IndexPattern.java	92.20%	3 Missing and 3 partials ⚠️
...h/security/privileges/PrivilegesConfiguration.java	81.25%	3 Missing ⚠️
...privileges/dlsfls/AbstractRuleBasedPrivileges.java	92.85%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5988      +/-   ##
==========================================
+ Coverage   73.81%   73.85%   +0.04%     
==========================================
  Files         439      440       +1     
  Lines       27122    27218      +96     
  Branches     4025     4038      +13     
==========================================
+ Hits        20020    20102      +82     
- Misses       5194     5205      +11     
- Partials     1908     1911       +3     

Files with missing lines	Coverage Δ
.../opensearch/security/OpenSearchSecurityPlugin.java	84.90% <100.00%> (-0.06%)	⬇️
...rch/security/configuration/DlsFlsRequestValve.java	0.00% <ø> (ø)
...search/security/configuration/DlsFlsValveImpl.java	64.89% <100.00%> (-0.37%)	⬇️
...earch/security/privileges/PrivilegesEvaluator.java	76.19% <ø> (ø)
...h/security/privileges/PrivilegesEvaluatorImpl.java	82.85% <100.00%> (ø)
...vileges/actionlevel/RoleBasedActionPrivileges.java	95.65% <100.00%> (+0.09%)	⬆️
...urity/privileges/actionlevel/WellKnownActions.java	85.71% <ø> (ø)
.../security/privileges/dlsfls/DlsFlsBaseContext.java	100.00% <100.00%> (ø)
...urity/privileges/dlsfls/DlsFlsProcessedConfig.java	100.00% <100.00%> (ø)
...security/privileges/dlsfls/DocumentPrivileges.java	91.22% <100.00%> (-0.58%)	⬇️
... and 7 more

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[nibix]

There seem to be test issues, taking back to draft

[nibix]

As an additional optimization, I have removed a few action names from the list of actions for which the denormalized data structures are created.

Reasoning:

• AutoCreate, AutoPut are likely never used because they work on non-existing indices. However the denormalized data structures by principle can only be created for existing indices
• Analyze, PutMapping: This is maybe too seldomly used to justify the overhead