Improve performance of the index selector

[V0ldek]

Is your feature request related to a problem? Please describe.
The index selector was introduced in #64, but there was no in-depth performance analysis and polishing applied afterwards. The potential for improvements may be large.

Describe the solution you'd like

Second, this requires an experimental approach. Run benches on the current code and then identify opportunities for improvement, run benchmarks again to verify an improvement, repeat. We will stop once we run out of improvement ideas.

[github-actions]

Tagging @V0ldek for notifications