TEZ-4689: Introduce Node abstraction for DAGAppMaster instead of sepagrate NodeManager-related fields#461
Open
Aggarwal-Raghav wants to merge 1 commit intoapache:masterfrom
Open
Conversation
Aggarwal-Raghav
commented
Feb 27, 2026
| private final int nmPort; | ||
| private final int nmHttpPort; | ||
| private String nmHost; | ||
| private int nmPort; |
Contributor
Author
There was a problem hiding this comment.
nmPort is not used anywhere but still keeping getAppNMPort() public api
Aggarwal-Raghav
commented
Feb 27, 2026
| if (!isLocal) { | ||
| this.nmHost = nodeContext.getNodeHostString(); | ||
| int nmHttpPort = Integer.parseInt(nodeContext.getNodeHttpPortString()); | ||
| this.containerLogs = |
Contributor
Author
There was a problem hiding this comment.
this.containerLogs is making use of nmHttpPort and nmHost, moved it from constructor to serviceInit() and made nmHttpPort as local variable
Aggarwal-Raghav
force-pushed
the
TEZ-4689
branch
from
c7923b1 to
fca17d7
Compare
Aggarwal-Raghav
commented
Feb 27, 2026
| DAGProtos.ConfigurationProto confProto = amExtensions.loadConfigurationProto(); | ||
| TezUtilsInternal.addUserSpecifiedTezConfiguration(conf, confProto.getConfKeyValuesList()); | ||
|
|
||
| NodeContext nodeContext = new YarnNodeManagerContext(); |
Contributor
Author
There was a problem hiding this comment.
I don't think we need to wrap in if statement for checking if framework is yarn. As the YarnNodeManagerContext is using supplier it won't be evaluated immediately and evaluation will happen only it !isLocal.
NOTE: in tez-am docker image tez.local.mode = true
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
Aggarwal-Raghav
commented
Feb 28, 2026
| TezConfiguration.TEZ_LOCAL_MODE_DEFAULT); | ||
|
|
||
| if (!isLocal) { | ||
| this.nmHost = nodeContext.getNodeHostString(); |
Contributor
Author
There was a problem hiding this comment.
This change is causing UT failues in TestMockDAGAppMaster.java because
is null post this change i.e. moving nmHost outside constructor. Working on fix...
Contributor
Author
There was a problem hiding this comment.
fixed. tested on local.
…rate NodeManager-related fields
Aggarwal-Raghav
force-pushed
the
TEZ-4689
branch
from
fca17d7 to
5e12682
Compare
|
🎊 +1 overall
This message was automatically generated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.