Can ChatGPT optimize a single line of Java Streams code?

Dear Skynet,

I meant no offense.

This article showcases an example in which ChatGPT 3.5 – March 14th, 2023 version fell short and eventually got very confused in my attempt to get it to optimize a single line of Java 8 Streams code.

While ChatGPT coming up with an incorrect solution to my problem, parts of the conversation include some fascinating (even when wrong) remarks from the chatbot, where you can tell that it gets close to tackling potential (although non-existent in this case).

The point of this experiment isn't to come up with a line similar to “Oh, ChatGPT can't refactor a single code statement.” Instead, I would say that even simple programming concepts can pose a significant challenge for AI software, at least for now.

What is ChatGPT?

ChatGPT is a powerful language model developed by OpenAI, based on the GPT-3.5 architecture. It is designed to understand natural language and generate human-like responses to various queries and conversations.

ChatGPT is trained on a massive corpus of text data and uses advanced machine learning algorithms to analyze and understand the context and meaning of input text. Its versatility and flexibility enable it to perform a wide range of tasks, from answering simple questions to carrying out complex conversations, and it can be integrated into various applications and platforms.

ChatGPT has gained widespread popularity due to its ability to produce high-quality, coherent responses that mimic human language and behavior, making it a valuable tool for businesses, researchers, and individuals seeking to communicate more effectively with machines.

How can ChatGPT help programmers?

I asked this very same question on ChatGPT and received the following quote:

ChatGPT is an incredible tool that can revolutionize the way programmers work. By integrating ChatGPT into their applications, programmers can create more intuitive and user-friendly interfaces that allow users to interact with their software in a more natural way. This can help improve the overall user experience and increase engagement with the software.

Additionally, ChatGPT can be used to automate tasks and provide helpful suggestions to programmers during the development process. By analyzing code and providing insights, ChatGPT can help programmers catch errors and identify areas for improvement, ultimately leading to better, more efficient code. Overall, ChatGPT is a valuable asset for any programmer looking to streamline their workflow and create more intuitive software.

ChatGPT

Consequently, I decided to put ChatGPT's refactoring capabilities to the test as I was writing a comprehensive Java 8 Streams tutorial. While ChatGPT did great in explaining Java Streams concepts, providing examples, and breaking them down, I wanted to see whether it can refactor a simple piece of code, which I came up with while writing about the importance of placing stream operators in the right order.

The Problem: Stream Operator Execution Order

The problem I came up with was a very simple case of Stream execution order. In particular, I wanted to produce a scenario where the Stream operation order can have an impact on performance.

I came up with the following two Stream statements:

List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5);

numbers.stream().filter(n -> n % 2 == 0)
                .map(n -> n * 2)
                .forEach(System.out::println); //Output: 4, 8

numbers.stream().map(n -> n * 2)
                .filter(n -> (n / 2) % 2 == 0)
                .forEach(System.out::println); //Output: 4, 8

While the above two statements produce the same results, the latter is problematic in terms of performance, as it will first perform the map operation across all data in the stream and then perform filtering.

A more efficient approach is to perform the filtering first and reduce the stream to just even numbers, then perform the map() operation on a potentially shorter stream, thus reducing execution time.

Simple enough, right?

Let's see whether ChatGPT could help me by coming up with something similar to the first statement.

Attempt #1: Explain What's Wrong

My Question

Explain what is wrong with the following Java 8 code:

List numbers = Arrays.asList(1, 2, 3, 4, 5);
Stream stream = numbers.stream().map(n -> n * 2).filter(n -> n / 2 % 2 == 0);

stream.forEach(System.out::println);

ChatGPT's Answer

The code creates a stream of even numbers by doubling each element in the list and filtering out the odd numbers. However, the condition used in the filter operation is incorrect, and it will not produce the desired output.

The condition n / 2 % 2 == 0 in the filter operation is problematic. When n is odd, n / 2 will be a non-integer value, and n / 2 % 2 will always be 1. Thus, the filter operation will exclude all odd numbers, but it will also include some even numbers that were not in the original list.

To fix the code, the condition in the filter operation should be changed to n % 2 == 0, which will correctly filter out all odd numbers.

The corrected code should be:

List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5);

Stream<Integer> stream = numbers.stream()
    .map(n -> n * 2)
    .filter(n -> n % 2 == 0);

stream.forEach(System.out::println);

This will produce the expected output of:

2
4
6
8
10

ChatGPT

ChatGPT's answer is wrong here, as it didn't figure out the problem I was trying to solve. While I was trying to filter the initial numbers, it instead produced a piece of code that filters the doubled numbers.

“When n is odd, n / 2 will be a non-integer value, and n / 2 % 2 will always be 1.” this is a wrong statement (e.g. 9 / 2 % 2). Even worse, the recommended code snippet indeed always produces even numbers, thus making the filter() operation completely redundant even if it wasn't wrong.

At the same time, using map(n -> n * 2) before filter(n -> n % 2 == 0) guarantees that all numbers will be even prior to division by 2, so all resulting numbers of the division will be the original integers.

ChatGPT will answer you that the result of n * 2 / 2 is n, but it could not successfully associate it with the concept of the two subsequent steam operations that lead to the same conclusion in terms of processing.

Attempt #2: Narrow Down the Problem

My Question

With my second question, I explicitly stated that the issue was related to the operation order to help ChatGPT figure it out.

Explain what is wrong with operator order in the following Java 8 Streams example:

List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5);

Stream<Integer> stream = numbers.stream()
                                                                .map(n -> n * 2)
                                                                .filter(n -> n / 2 % 2 == 0); stream.forEach(System.out::println);

ChatGPT

ChatGPT's Answer

There is no issue with the operator order in this Java 8 Streams example. The map() operator is applied first to each element in the list to double its value, and then the filter() operator is applied to only keep elements whose integer division by 2 and subsequent integer division by 2 again results in 0 (i.e. it keeps only the even numbers). Finally, the forEach() operator is called to print the resulting stream of even numbers.

What I realized here is that maybe using the word “wrong” was a mistake, as ChatGPT was probably looking for a bug instead of a performance optimization opportunity.

Attempt #3: Providing Context (i.e., a correct answer)

My Question

In my third attempt, I decided to add more context by providing the correct “statement” and asking the chatbot why the second statement is suboptimal.

Describe why the second statement in the code below is suboptimal:

numbers.stream()
                .filter(n -> n % 2 == 0)
                .map(n -> n * 2)
                .forEach(System.out::println);

numbers.stream()
                .map(n -> n * 2)
                .filter(n -> (n / 2) % 2 == 0)
                .forEach(System.out::println);

This led to a correct response.

ChatGPT's Answer

In the given code, the second statement is suboptimal because it involves more computation than the first statement.

In the first statement, the filter() operation is applied before the map() operation. This is beneficial because the filter() operation eliminates half of the elements in the stream before the map() operation doubles the remaining elements. Therefore, the map() operation will only have to perform half the number of computations it would have to perform if it was applied to all the elements in the stream.

On the other hand, the second statement applies the map() operation before the filter() operation. This means that the map() operation will be applied to all elements in the stream, even though some of them will be eliminated by the filter() operation. This results in unnecessary computation, which can slow down the performance of the code.

Therefore, it is recommended to apply the filter() operation before the map() operation to optimize the performance of the code.

ChatGPT

ChatGPT identified the issue and explained it with much better wording than I ever could. That's great, but can it improve the code? Let's see!

Attempt #4: Asking for Improvement

My Question

Improve the following code:

numbers.stream().map(n -> n * 2) 
                                .filter(n -> (n / 2) % 2 == 0)
                                .forEach(System.out::println); //Output: 4, 8

ChatGPT's Answer

The given code can be improved by using the collect method instead of forEach to collect the filtered and mapped elements into a new list. Here's the improved code:

List<Integer> result = numbers.stream()
                                                       .map(n -> n * 2)
                                                       .filter(n -> (n / 2) % 2 == 0)
                                                       .collect(Collectors.toList());

System.out.println(result); // Output: [4, 8]

In this code, the collect method is used to collect the filtered and mapped elements into a new list. The Collectors.toList() method is used to collect the elements into a list. Finally, the System.out.println statement is used to print the list.

ChatGPT

Another mistake on my end here is that I did not specifically ask for a performance improvement. Collecting the results into a List is by no means unreasonable, but that wasn't what I was looking for.

Attempt #5: Ask for Performance Improvement

My Question

Improve the performance of the following code:

numbers.stream().map(n -> n * 2) 
                .filter(n -> (n / 2) % 2 == 0)
                .forEach(System.out::println); //Output: 4, 8

ChatGPT's Answer

To improve the performance of this code, we can use the parallel stream instead of the sequential stream. Here is an example:

numbers.parallelStream()
                .map(n -> n * 2)
                .filter(n -> (n / 2) % 2 == 0)
                .forEach(System.out::println); // Output: 4, 8

By using parallel streams, we can take advantage of multi-core CPUs and increase the speed of our processing, especially for large datasets. However, it's important to note that parallel streams may not always be faster and may not be appropriate for certain types of operations, so it's important to test and benchmark different approaches to determine the best performance for your specific use case.

This answer is correct albeit suggesting a very obvious performance improvement. One could argue that switching the filter() and map() operations is also an obvious improvement, so I expected more here.

Once we reached that point in the conversation I decided to ask for more improvements for my piece of code and see everything ChatGPT can come up with.

Attempt #6: Recommend More Improvements

My Question

Can you recommend more improvements for:

numbers.stream().map(n -> n * 2)                               
                                .filter(n -> (n / 2) % 2 == 0)
                               .forEach(System.out::println); 

ChatGPT's Answer

Yes, here are some further improvements that can be made to this code:

1. Use parallel streams: If the data set is large enough, using parallel streams can improve performance by leveraging multiple threads for processing. To do this, simply replace stream() with parallelStream().
2. Use method reference instead of lambda: Instead of using a lambda expression to multiply each number by 2, you can use a method reference to the Integer class's intValue() method, like so: mapToInt(Integer::intValue). This can sometimes result in slightly faster code.
3. Use filter before map: Since the filter operation reduces the size of the data set, it can be more efficient to perform it before the map operation. This reduces the number of elements that need to be mapped, resulting in faster code.

Here is an updated version of the code that incorporates these improvements:

numbers.parallelStream()
                .filter(n -> n % 2 == 0)
                .mapToInt(Integer::intValue)
                .filter(n -> n % 4 == 0)
                .forEach(System.out::println);

This code first filters out all odd numbers, then maps the remaining even numbers to their integer values, and finally filters out any numbers that are not divisible by 4 before printing the result.

ChatGPT

While 2 out of ChatGPT's recommendations make sense here, the resulting code is awfully wrong, resulting in a stream that keeps the substream of original numbers that are divisible by 4.

What Went Wrong

The best way to describe the problem here is that ChatGPT could not follow the combination of arithmetic expressions and stream operations. I avoid using the term “understand” for this kind of AI. It rather seems to try to infer what we are trying to do based on what it's seen unless its knowledge base includes identical examples.

In this example, ChatGPT did not “understand” what I was trying to do with my code and tried to solve a different problem. Its last answer may imply that it may get confused with simple arithmetic operations when they have combined with another, more complex concept even though Streams have pretty concrete semantics. The last filtering condition n -> n % 4 == 0 is probably a consequence of not understanding how stream operators affect streams, rather than ChatGPT messing up operation priorities and equivalencies.