Post Syndicated from Jonathan Vogel original https://aws.amazon.com/blogs/devops/dissecting-the-performance-gains-in-amazon-q-developer-agent-for-code-transformation/

Amazon Q Developer Agent for code transformation is an AI-powered tool which modernizes code bases from Java 8 and Java 11 to Java 17. Integrated into VS Code and IntelliJ, Amazon Q simplifies the migration process and reduce the time and effort compared to manual process. It proposes and verifies code changes, using AI to debug compilation errors. In this blog post, we’ll explore recent improvements to our code transformation agent, particularly its enhanced debugging capabilities. The enhanced debugger agent significantly improves transformation efficiency and quality compared to the existing debugger.

How Amazon Q transforms Java applications

To upgrade Java codebases, the code transformation agent takes the source code input and verify the build and test in source Java version. It then uses deterministic tools to apply code changes, followed by building and testing the changed code in the target Java version. If errors occur in this stage, a generative AI-based system debugs and resolves the compilation errors. Until today, the debugger resolves each error one by one, locating the code file with the error in the codebase, and fixing it. This debug step iterates until all compilation errors are solved or the maximum number of iterations is reached.

As an example, if, as the result of a library upgrade, an import statement is missing or wrong, the AI debugger will re-build, iterate to find all the references in multiple files one by one, and update each reference to resolve the error. Refer to this blog “Three ways Amazon Q Developer agent for code transformation accelerates Java upgrades” for detailed explanation of each transformation step. This approach has helped Q Developer customers achieve accelerations of migration effort by over 40%.

Improving the debugging capabilities of code transformations

To further improve the ability of Q Developer to generate error-free code, we’ve just released multiple foundational improvements to the AI debugger.

• Multi-error context: the debug AI can now take multiple build errors into consideration, which provides more context, leading to better solution discovery.
• More tools available for the AI: compared to simply localizing error to a single file and fixing the error previously, the agent can now execute multi-file solutions by exploring the codebase and operating on multiple files.
• Inter-iteration memory: the debugger AI now remembers previous errors, which contributes to debugging new errors.
• Intelligent backtracking: the debugger AI can now recognize if the current solution path leads to a dead end, in which case the agent can roll back to the previous state.

To implement these capabilities, the debugger AI is re-architected as a multi-agent system. A memory management agent is responsible to analyze last iteration results and append the relevant portions to the inter-iteration memory. A critic agent is responsible to analyze progress and provide additional information to the debugger agent and, if a dead end is detected, rollback the progress to a previous state. A debugger agent, analyzes the memory and the critique from the previous agents and modifies or updates the plan to fix the remaining errors in the codebase. The debugger agent has its disposal a set of generic and specialized tools to browse and explore the codebase, edit source files, trigger builds, add dependencies, and so on. It is important to note that the agent only has access to the files and tools related to the transformation task, which limits hallucinations and drive towards progress.

Let’s examine how the agent handles recurring issues across multiple files with these improvements. Consider a scenario where several Java files are missing the same import statement after upgrading from Java 8 to Java 17. This happens when you upgrade from older Java collections (like Vector and Enumeration) to modern streaming operations. The system is capable of helping you update these patterns automatically. The agent is now able to intelligently detect this pattern and implement a comprehensive solution across all affected files. Suppose we have three Java files that use the java.util.stream.Collectors class, but the import is missing in each:

File1.java:

public class File1 {
    public List<String> process(List<String> input) {
        return input.stream()
            .filter(s → s.length() > 5)
            .collect(Collectors.toList()); // Error: Cannot resolve symbol 'Collectors'
    }
}

File2.java:

public class File2 {
    public Map<String, Long> countWords(List<String> words) {
        return words.stream()
            .collect(Collectors.groupingBy(
                word -> word.toLowerCase(),
                Collectors.counting()
            )); // Error: Cannot resolve symbol 'Collectors'
    }
}

File3.java:

public class File3 {
    public String concatenate(List<String> strings) {
        return strings.stream()
            .collect(Collectors.joining(", "));
            // Error: Cannot resolve symbol 'Collectors'
    }
}

After the agent detects the common issue and applies the fix, all three files would be updated as follows:

File1.java (after fix):

import java.util.stream.Collectors;

public class File1 {
    public List<String> process(List<String> input) {
        return input.stream()
            .filter(s -> s.length() > 5)
            .collect(Collectors.toList());
    }
}    

File2.java (after fix):

import java.util.stream.Collectors;

public class File2 {
    public Map<String, Long> countWords(List<String> words) {
        return words.stream()
            .collect(Collectors.groupingBy(
                word -> word.toLowerCase(),
                Collectors.counting()));
    }
}

File3.java (after fix):

import java.util.stream.Collectors;

public class File3 {
    public String concatenate(List<String> strings) {
        return strings.stream()
            .collect(Collectors.joining(", "));
    }
}

In this example, the agent has identified that the same import statement (import java.util.stream.Collectors;) was missing in all three files. It then applied the fix consistently across all affected files, demonstrating its ability to recognize patterns and implement solutions efficiently across the entire codebase, avoiding different solutions attempts for each individual error, and saving iteration budget to solve different errors, if present.

The contrast between existing debugger and enhanced Agent is more clear when handling complex, interconnected changes. For instance, in updating Springfox Swagger from 2.0 to 3.0 (OpenAPI), both systems initially made similar changes. However, when faced with subsequent errors, their approaches diverged significantly. Consider this scenario:
Initially, both systems removed Springfox dependencies:

<!-- Removed by both systems -->
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger2</artifactId>
    <version>2.9.2</version>
</dependency>

Later, when encountering a “missing symbol: Docket” error, existing debugger attempted to reintroduce Springfox:

<!-- existing debugger trying to add back Springfox -->
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-boot-starter</artifactId>
    <version>3.0.0</version>
</dependency>

In contrast, our Agent recognized this as consistent with the previous removal and rewrote the file using SpringDoc OpenAPI:

import org.springdoc.core.GroupedOpenApi;

@Configuration
public class SwaggerConfig {
    @Bean
    public GroupedOpenApi publicApi() {
        return GroupedOpenApi.builder()
                .group("springshop-public")
                .pathsToMatch("/public/**")
                .build();
    }
}   

These latest improvements in our debug AI have yielded positive results. By incorporating multi-error context analysis, additional tooling of multi-file solution, and inter-iteration memory, the agent now delivers more comprehensive and consistent codebase upgrades. We tested our new approach on 62 large open-source applications, some containing over 100,000 lines of code, incorporating more than 100 open-source libraries. The results showed an 85% higher success rate compared to the previous approach. These enhancements significantly boost both the quality and efficiency of code transformation, marking a substantial leap forward in automated application modernization for Java.

Conclusion

With the latest improvements, Q Developer continues to accelerate the journey to modernize Java applications across your organization. For more context, please refer to the blog “Accelerate application upgrades with Amazon Q Developer agent for code transformation.”

As we continue to innovate in code transformation use cases, this release creates the foundation to expand language support, further enhance AI-driven problem-solving algorithms, and streamlining the integration with development workflows. Our goal remains to provide developers and organizations with cutting-edge tools that simplify complex maintenance and modernization processes and foster the adoption of modern, cloud-native architectures. Stay tuned for future updates as we push the boundaries of AI-assisted code transformation.

About the authors