Four calls to `Parser.parse` cause performance issues

[TomerAberbach]

Hey! We've been using ktfmt for our generated SDKs at Stainless. We've found the formatter can be slow sometimes due to the size of the SDKs we generate (the larger the API, the larger the generated SDK).

I decided to do some profiling and I found Parser.parse takes a significant amount of time (ktfmt-flamegraph.html.zip attached for running anthropic-sdk-java) because it's called four times per file.

Do you think it's possible to ensure each file gets parsed exactly once? I recognize it's not a trivial problem because subsequent transformations would somehow need to take into account edits from previous transformations, but seems like a worthwhile investment.

[hick209]

Thanks a ton for sharing your use case and perf issues you have faced.

At the moment this is not a current priority for us so it might take a while to get to it, but we would certainly welcome PRs around this.

[abinesh08]

Hai TomerAberbach,

Thanks for sharing your observations and the flamegraph. I completely understand your concern about Parser.parse being called multiple times per file — it can definitely become a bottleneck for large SDKs.

In simple terms, each time Parser.parse is called, the formatter has to read and convert the Kotlin file into an internal syntax tree. If this happens multiple times, it naturally slows things down, especially for large files. Ideally, we would want each file to be parsed exactly once, and then all transformations or formatting steps should operate on that same parsed structure.

The challenge here, as you mentioned, is that later transformations may depend on changes made by earlier ones. So we need a system where:

We parse the file once into a syntax tree.

Each transformation or formatting step works on that tree, updating it in memory rather than reparsing the file.

At the end, the tree is converted back into formatted Kotlin code.

This approach would reduce redundant parsing and should noticeably improve performance for large SDKs. It’s definitely a non-trivial change but seems like a worthwhile optimization, especially for generated SDKs where files can be very large.

yes, it’s possible, but it would require restructuring how transformations are applied to ensure they all work on the same parsed representation.

[nreid260]

I looked into this like a year ago and my conclusion was that it wasn't worth it.

The parser produces PSI, which is actually designed for syntax editing in an IDE context, so you'd expect it to work great for this. Unfortunately, the PSI assumes that huge other chunks of IntelliJ code are present when using the syntax updating functionality, which kftmt doesn't pull in. The result is that supplying/faking these IntelliJ deps spirals out into a dynamic loading mess.

On top of the PSI stuff, we're also handcuffed by using GJF, which we want for it's layout engine. Using that engine requires that the code be printed, but we want to do more AST processing after it runs, so we end up parsing again. It would be a major redesign to get this to work any other way.

---

Another approach to your underlying problem would be to make ktfmt faster in general. We've had success with using Graal to make a native image of ktfmt, which is an incredible speedup. There are tradeoffs in executable size and dev cycle, but maybe it would be a solution for you.