Skip to content

Remove unneed allocation#60

Open
roj234 wants to merge 2 commits intoJamePeng:mainfrom
roj234:main
Open

Remove unneed allocation#60
roj234 wants to merge 2 commits intoJamePeng:mainfrom
roj234:main

Conversation

@roj234
Copy link

@roj234 roj234 commented Feb 15, 2026

It seems that self.scores only changed when self._logits_all == True, so only allocate it when self._logits_all == True

@JamePeng
Copy link
Owner

You raised a good point. I've noticed that the old logic for loading and reading state in the original llama-cpp-python has deviated from the way llama.cpp operates. I'll probably add this to my to-do list to refactor this part of the logic.

You could also add a null check on the score array before it's called.

@roj234
Copy link
Author

roj234 commented Feb 15, 2026

Unfortunately, (since I'm only used it to create embedding), I have not noticed that self.scores was used in generate (so it really need more refactors...)

@roj234 roj234 closed this Feb 15, 2026
@roj234 roj234 reopened this Feb 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants