Out of memory error in Kotlin when looping through a medium size data set












1















I am running the below loop in Kotlin and is throwing an out of memory error. I am running this for reading the rows in a csv file. Size of "records" is 6422.



fun readCSVFile(filePath: String): List<String> {
val reader = FileReader(filePath)
val records = CSVFormat.DEFAULT.parse(reader)
val rows = mutableListOf<String>()

var output = ""
records.forEach() {
val size = it.size()
for (i in 0 until it.size()-1) {
output = output + it.get(i) + ","
}
output.dropLast(1)
rows.add(output)
}
return rows
}


Below is the exception I get.



Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
at java.lang.StringBuilder.append(StringBuilder.java:136)
at trivago.ti.tools.FileProcessor.readCSVFile(FileProcessor.kt:16)
at trivago.ti.tools.ComparatorMainKt.main(ComparatorMain.kt:25)


I have the same logic executing in Java but it works fine. Below is what I have in Java.



private static List<String> readCSVFile(String filePath) throws IOException {
Reader in = new FileReader(filePath);
Iterable<CSVRecord> records = CSVFormat.DEFAULT.parse(in);
List<String> rows = new ArrayList<>();
for (CSVRecord record : records) {
StringBuilder builder = new StringBuilder();
for (int i = 0; i < record.size(); i++)
builder.append(record.get(i) + ",");
builder.deleteCharAt(builder.length() - 1);
rows.add(builder.toString());
}
return rows;
}


Why is Kotlin having a problem with this? Am I doing something wrong with the loop? Any help would be much appreciated as I'm a newbie to Kotlin.










share|improve this question

























  • What size is the CSV you are reading? What's the -Xmx value of your JVM (if you defined any yourself) ?

    – Sofo Gial
    Nov 14 '18 at 14:33











  • Sorry I edited the question. its 6422

    – mayooran
    Nov 14 '18 at 14:34











  • In bytes? Also, did you define the -Xmx anywhere or is it default?

    – Sofo Gial
    Nov 14 '18 at 14:36











  • I guess the answers have solved the problem, so this is to solve another one: replace i in 0 until it.size()-1 with i in 0 until it.size() or i in 0..it.size()-1 because the code as it is loses the last line.

    – forpas
    Nov 14 '18 at 14:58
















1















I am running the below loop in Kotlin and is throwing an out of memory error. I am running this for reading the rows in a csv file. Size of "records" is 6422.



fun readCSVFile(filePath: String): List<String> {
val reader = FileReader(filePath)
val records = CSVFormat.DEFAULT.parse(reader)
val rows = mutableListOf<String>()

var output = ""
records.forEach() {
val size = it.size()
for (i in 0 until it.size()-1) {
output = output + it.get(i) + ","
}
output.dropLast(1)
rows.add(output)
}
return rows
}


Below is the exception I get.



Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
at java.lang.StringBuilder.append(StringBuilder.java:136)
at trivago.ti.tools.FileProcessor.readCSVFile(FileProcessor.kt:16)
at trivago.ti.tools.ComparatorMainKt.main(ComparatorMain.kt:25)


I have the same logic executing in Java but it works fine. Below is what I have in Java.



private static List<String> readCSVFile(String filePath) throws IOException {
Reader in = new FileReader(filePath);
Iterable<CSVRecord> records = CSVFormat.DEFAULT.parse(in);
List<String> rows = new ArrayList<>();
for (CSVRecord record : records) {
StringBuilder builder = new StringBuilder();
for (int i = 0; i < record.size(); i++)
builder.append(record.get(i) + ",");
builder.deleteCharAt(builder.length() - 1);
rows.add(builder.toString());
}
return rows;
}


Why is Kotlin having a problem with this? Am I doing something wrong with the loop? Any help would be much appreciated as I'm a newbie to Kotlin.










share|improve this question

























  • What size is the CSV you are reading? What's the -Xmx value of your JVM (if you defined any yourself) ?

    – Sofo Gial
    Nov 14 '18 at 14:33











  • Sorry I edited the question. its 6422

    – mayooran
    Nov 14 '18 at 14:34











  • In bytes? Also, did you define the -Xmx anywhere or is it default?

    – Sofo Gial
    Nov 14 '18 at 14:36











  • I guess the answers have solved the problem, so this is to solve another one: replace i in 0 until it.size()-1 with i in 0 until it.size() or i in 0..it.size()-1 because the code as it is loses the last line.

    – forpas
    Nov 14 '18 at 14:58














1












1








1








I am running the below loop in Kotlin and is throwing an out of memory error. I am running this for reading the rows in a csv file. Size of "records" is 6422.



fun readCSVFile(filePath: String): List<String> {
val reader = FileReader(filePath)
val records = CSVFormat.DEFAULT.parse(reader)
val rows = mutableListOf<String>()

var output = ""
records.forEach() {
val size = it.size()
for (i in 0 until it.size()-1) {
output = output + it.get(i) + ","
}
output.dropLast(1)
rows.add(output)
}
return rows
}


Below is the exception I get.



Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
at java.lang.StringBuilder.append(StringBuilder.java:136)
at trivago.ti.tools.FileProcessor.readCSVFile(FileProcessor.kt:16)
at trivago.ti.tools.ComparatorMainKt.main(ComparatorMain.kt:25)


I have the same logic executing in Java but it works fine. Below is what I have in Java.



private static List<String> readCSVFile(String filePath) throws IOException {
Reader in = new FileReader(filePath);
Iterable<CSVRecord> records = CSVFormat.DEFAULT.parse(in);
List<String> rows = new ArrayList<>();
for (CSVRecord record : records) {
StringBuilder builder = new StringBuilder();
for (int i = 0; i < record.size(); i++)
builder.append(record.get(i) + ",");
builder.deleteCharAt(builder.length() - 1);
rows.add(builder.toString());
}
return rows;
}


Why is Kotlin having a problem with this? Am I doing something wrong with the loop? Any help would be much appreciated as I'm a newbie to Kotlin.










share|improve this question
















I am running the below loop in Kotlin and is throwing an out of memory error. I am running this for reading the rows in a csv file. Size of "records" is 6422.



fun readCSVFile(filePath: String): List<String> {
val reader = FileReader(filePath)
val records = CSVFormat.DEFAULT.parse(reader)
val rows = mutableListOf<String>()

var output = ""
records.forEach() {
val size = it.size()
for (i in 0 until it.size()-1) {
output = output + it.get(i) + ","
}
output.dropLast(1)
rows.add(output)
}
return rows
}


Below is the exception I get.



Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
at java.lang.StringBuilder.append(StringBuilder.java:136)
at trivago.ti.tools.FileProcessor.readCSVFile(FileProcessor.kt:16)
at trivago.ti.tools.ComparatorMainKt.main(ComparatorMain.kt:25)


I have the same logic executing in Java but it works fine. Below is what I have in Java.



private static List<String> readCSVFile(String filePath) throws IOException {
Reader in = new FileReader(filePath);
Iterable<CSVRecord> records = CSVFormat.DEFAULT.parse(in);
List<String> rows = new ArrayList<>();
for (CSVRecord record : records) {
StringBuilder builder = new StringBuilder();
for (int i = 0; i < record.size(); i++)
builder.append(record.get(i) + ",");
builder.deleteCharAt(builder.length() - 1);
rows.add(builder.toString());
}
return rows;
}


Why is Kotlin having a problem with this? Am I doing something wrong with the loop? Any help would be much appreciated as I'm a newbie to Kotlin.







java memory kotlin jvm out-of-memory






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 14 '18 at 14:34







mayooran

















asked Nov 14 '18 at 14:29









mayooranmayooran

1,68432858




1,68432858













  • What size is the CSV you are reading? What's the -Xmx value of your JVM (if you defined any yourself) ?

    – Sofo Gial
    Nov 14 '18 at 14:33











  • Sorry I edited the question. its 6422

    – mayooran
    Nov 14 '18 at 14:34











  • In bytes? Also, did you define the -Xmx anywhere or is it default?

    – Sofo Gial
    Nov 14 '18 at 14:36











  • I guess the answers have solved the problem, so this is to solve another one: replace i in 0 until it.size()-1 with i in 0 until it.size() or i in 0..it.size()-1 because the code as it is loses the last line.

    – forpas
    Nov 14 '18 at 14:58



















  • What size is the CSV you are reading? What's the -Xmx value of your JVM (if you defined any yourself) ?

    – Sofo Gial
    Nov 14 '18 at 14:33











  • Sorry I edited the question. its 6422

    – mayooran
    Nov 14 '18 at 14:34











  • In bytes? Also, did you define the -Xmx anywhere or is it default?

    – Sofo Gial
    Nov 14 '18 at 14:36











  • I guess the answers have solved the problem, so this is to solve another one: replace i in 0 until it.size()-1 with i in 0 until it.size() or i in 0..it.size()-1 because the code as it is loses the last line.

    – forpas
    Nov 14 '18 at 14:58

















What size is the CSV you are reading? What's the -Xmx value of your JVM (if you defined any yourself) ?

– Sofo Gial
Nov 14 '18 at 14:33





What size is the CSV you are reading? What's the -Xmx value of your JVM (if you defined any yourself) ?

– Sofo Gial
Nov 14 '18 at 14:33













Sorry I edited the question. its 6422

– mayooran
Nov 14 '18 at 14:34





Sorry I edited the question. its 6422

– mayooran
Nov 14 '18 at 14:34













In bytes? Also, did you define the -Xmx anywhere or is it default?

– Sofo Gial
Nov 14 '18 at 14:36





In bytes? Also, did you define the -Xmx anywhere or is it default?

– Sofo Gial
Nov 14 '18 at 14:36













I guess the answers have solved the problem, so this is to solve another one: replace i in 0 until it.size()-1 with i in 0 until it.size() or i in 0..it.size()-1 because the code as it is loses the last line.

– forpas
Nov 14 '18 at 14:58





I guess the answers have solved the problem, so this is to solve another one: replace i in 0 until it.size()-1 with i in 0 until it.size() or i in 0..it.size()-1 because the code as it is loses the last line.

– forpas
Nov 14 '18 at 14:58












3 Answers
3






active

oldest

votes


















3














Use a StringBuilder in your kotlin code as well. You are creating a log of String objects in heap. String is immutable and this code:



var output = ""
output = output + ","


is creating two objects in heap, although you have only reference to one of them. So the other one is eligible for GC to remove it. GC is "working" too hard in your case, that's why you get java.lang.OutOfMemoryError: GC overhead limit exceeded.



fun readCSVFile(filePath: String): List<String> {
val reader = FileReader(filePath)
val records = CSVFormat.DEFAULT.parse(reader)
val rows = mutableListOf<String>()

var output = StringBuilder("")
records.forEach() {
output = StringBuilder("")
val size = it.size()
for (i in 0 until it.size()-1) {
output = output.append(it.get(i) + ",")
}
output.deleteCharAt(output.length - 1)
rows.add(output.toString())
}
return rows
}


You code will also run a lot faster, as creating a new object is quite costly.






share|improve this answer

































    4














    I think there is a bug in your code



    records.forEach() {
    output = "" // clear output ;)
    ...
    }


    Compare it to your java code



    for (CSVRecord record : records) {
    StringBuilder builder = new StringBuilder(); // clear builder
    ...
    }





    share|improve this answer


























    • Did you try this ? I don't think this will help, as OP is getting GC overhead limit exceeded, that means there are a lot of objects in heap

      – Schidu Luca
      Nov 14 '18 at 15:05











    • I agree that StringBuilder will help, but this will also help, because this is the cause of the GC having to run that many times. rows.add(output) -> rows.add("row1"); rows.add(output) -> rows.add("row1row2"); rows.add(output) -> rows.add("row1row2row3"); rows.add(output) -> rows.add("row1row2row3...row6422");

      – Julio Daniel Reyes
      Nov 14 '18 at 15:28













    • this does not affect in any way how many time GC runs rows.add(output), the string just keeps getting bigger, OP's mistake

      – Schidu Luca
      Nov 14 '18 at 15:37











    • The string keeps getting bigger, running out of heap memory, will cause the GC to run. GC has been trying to free the memory but is pretty much unable to get any job done, as rows content cannot be garbage collected.

      – Julio Daniel Reyes
      Nov 14 '18 at 15:44



















    0














    You have two problems in your Kotlin code:




    1. You are using strings and string concatenation - which is costly operation. You should use StringBuilder as well.

    2. You are setting output = "" outside of the foreach loop - for each iteration, you have all previous rows inside output






    share|improve this answer























      Your Answer






      StackExchange.ifUsing("editor", function () {
      StackExchange.using("externalEditor", function () {
      StackExchange.using("snippets", function () {
      StackExchange.snippets.init();
      });
      });
      }, "code-snippets");

      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "1"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: true,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: 10,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53302540%2fout-of-memory-error-in-kotlin-when-looping-through-a-medium-size-data-set%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      3 Answers
      3






      active

      oldest

      votes








      3 Answers
      3






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      3














      Use a StringBuilder in your kotlin code as well. You are creating a log of String objects in heap. String is immutable and this code:



      var output = ""
      output = output + ","


      is creating two objects in heap, although you have only reference to one of them. So the other one is eligible for GC to remove it. GC is "working" too hard in your case, that's why you get java.lang.OutOfMemoryError: GC overhead limit exceeded.



      fun readCSVFile(filePath: String): List<String> {
      val reader = FileReader(filePath)
      val records = CSVFormat.DEFAULT.parse(reader)
      val rows = mutableListOf<String>()

      var output = StringBuilder("")
      records.forEach() {
      output = StringBuilder("")
      val size = it.size()
      for (i in 0 until it.size()-1) {
      output = output.append(it.get(i) + ",")
      }
      output.deleteCharAt(output.length - 1)
      rows.add(output.toString())
      }
      return rows
      }


      You code will also run a lot faster, as creating a new object is quite costly.






      share|improve this answer






























        3














        Use a StringBuilder in your kotlin code as well. You are creating a log of String objects in heap. String is immutable and this code:



        var output = ""
        output = output + ","


        is creating two objects in heap, although you have only reference to one of them. So the other one is eligible for GC to remove it. GC is "working" too hard in your case, that's why you get java.lang.OutOfMemoryError: GC overhead limit exceeded.



        fun readCSVFile(filePath: String): List<String> {
        val reader = FileReader(filePath)
        val records = CSVFormat.DEFAULT.parse(reader)
        val rows = mutableListOf<String>()

        var output = StringBuilder("")
        records.forEach() {
        output = StringBuilder("")
        val size = it.size()
        for (i in 0 until it.size()-1) {
        output = output.append(it.get(i) + ",")
        }
        output.deleteCharAt(output.length - 1)
        rows.add(output.toString())
        }
        return rows
        }


        You code will also run a lot faster, as creating a new object is quite costly.






        share|improve this answer




























          3












          3








          3







          Use a StringBuilder in your kotlin code as well. You are creating a log of String objects in heap. String is immutable and this code:



          var output = ""
          output = output + ","


          is creating two objects in heap, although you have only reference to one of them. So the other one is eligible for GC to remove it. GC is "working" too hard in your case, that's why you get java.lang.OutOfMemoryError: GC overhead limit exceeded.



          fun readCSVFile(filePath: String): List<String> {
          val reader = FileReader(filePath)
          val records = CSVFormat.DEFAULT.parse(reader)
          val rows = mutableListOf<String>()

          var output = StringBuilder("")
          records.forEach() {
          output = StringBuilder("")
          val size = it.size()
          for (i in 0 until it.size()-1) {
          output = output.append(it.get(i) + ",")
          }
          output.deleteCharAt(output.length - 1)
          rows.add(output.toString())
          }
          return rows
          }


          You code will also run a lot faster, as creating a new object is quite costly.






          share|improve this answer















          Use a StringBuilder in your kotlin code as well. You are creating a log of String objects in heap. String is immutable and this code:



          var output = ""
          output = output + ","


          is creating two objects in heap, although you have only reference to one of them. So the other one is eligible for GC to remove it. GC is "working" too hard in your case, that's why you get java.lang.OutOfMemoryError: GC overhead limit exceeded.



          fun readCSVFile(filePath: String): List<String> {
          val reader = FileReader(filePath)
          val records = CSVFormat.DEFAULT.parse(reader)
          val rows = mutableListOf<String>()

          var output = StringBuilder("")
          records.forEach() {
          output = StringBuilder("")
          val size = it.size()
          for (i in 0 until it.size()-1) {
          output = output.append(it.get(i) + ",")
          }
          output.deleteCharAt(output.length - 1)
          rows.add(output.toString())
          }
          return rows
          }


          You code will also run a lot faster, as creating a new object is quite costly.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 14 '18 at 15:02

























          answered Nov 14 '18 at 14:49









          Schidu LucaSchidu Luca

          2,834520




          2,834520

























              4














              I think there is a bug in your code



              records.forEach() {
              output = "" // clear output ;)
              ...
              }


              Compare it to your java code



              for (CSVRecord record : records) {
              StringBuilder builder = new StringBuilder(); // clear builder
              ...
              }





              share|improve this answer


























              • Did you try this ? I don't think this will help, as OP is getting GC overhead limit exceeded, that means there are a lot of objects in heap

                – Schidu Luca
                Nov 14 '18 at 15:05











              • I agree that StringBuilder will help, but this will also help, because this is the cause of the GC having to run that many times. rows.add(output) -> rows.add("row1"); rows.add(output) -> rows.add("row1row2"); rows.add(output) -> rows.add("row1row2row3"); rows.add(output) -> rows.add("row1row2row3...row6422");

                – Julio Daniel Reyes
                Nov 14 '18 at 15:28













              • this does not affect in any way how many time GC runs rows.add(output), the string just keeps getting bigger, OP's mistake

                – Schidu Luca
                Nov 14 '18 at 15:37











              • The string keeps getting bigger, running out of heap memory, will cause the GC to run. GC has been trying to free the memory but is pretty much unable to get any job done, as rows content cannot be garbage collected.

                – Julio Daniel Reyes
                Nov 14 '18 at 15:44
















              4














              I think there is a bug in your code



              records.forEach() {
              output = "" // clear output ;)
              ...
              }


              Compare it to your java code



              for (CSVRecord record : records) {
              StringBuilder builder = new StringBuilder(); // clear builder
              ...
              }





              share|improve this answer


























              • Did you try this ? I don't think this will help, as OP is getting GC overhead limit exceeded, that means there are a lot of objects in heap

                – Schidu Luca
                Nov 14 '18 at 15:05











              • I agree that StringBuilder will help, but this will also help, because this is the cause of the GC having to run that many times. rows.add(output) -> rows.add("row1"); rows.add(output) -> rows.add("row1row2"); rows.add(output) -> rows.add("row1row2row3"); rows.add(output) -> rows.add("row1row2row3...row6422");

                – Julio Daniel Reyes
                Nov 14 '18 at 15:28













              • this does not affect in any way how many time GC runs rows.add(output), the string just keeps getting bigger, OP's mistake

                – Schidu Luca
                Nov 14 '18 at 15:37











              • The string keeps getting bigger, running out of heap memory, will cause the GC to run. GC has been trying to free the memory but is pretty much unable to get any job done, as rows content cannot be garbage collected.

                – Julio Daniel Reyes
                Nov 14 '18 at 15:44














              4












              4








              4







              I think there is a bug in your code



              records.forEach() {
              output = "" // clear output ;)
              ...
              }


              Compare it to your java code



              for (CSVRecord record : records) {
              StringBuilder builder = new StringBuilder(); // clear builder
              ...
              }





              share|improve this answer















              I think there is a bug in your code



              records.forEach() {
              output = "" // clear output ;)
              ...
              }


              Compare it to your java code



              for (CSVRecord record : records) {
              StringBuilder builder = new StringBuilder(); // clear builder
              ...
              }






              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Nov 14 '18 at 15:03

























              answered Nov 14 '18 at 14:41









              Julio Daniel ReyesJulio Daniel Reyes

              2,082815




              2,082815













              • Did you try this ? I don't think this will help, as OP is getting GC overhead limit exceeded, that means there are a lot of objects in heap

                – Schidu Luca
                Nov 14 '18 at 15:05











              • I agree that StringBuilder will help, but this will also help, because this is the cause of the GC having to run that many times. rows.add(output) -> rows.add("row1"); rows.add(output) -> rows.add("row1row2"); rows.add(output) -> rows.add("row1row2row3"); rows.add(output) -> rows.add("row1row2row3...row6422");

                – Julio Daniel Reyes
                Nov 14 '18 at 15:28













              • this does not affect in any way how many time GC runs rows.add(output), the string just keeps getting bigger, OP's mistake

                – Schidu Luca
                Nov 14 '18 at 15:37











              • The string keeps getting bigger, running out of heap memory, will cause the GC to run. GC has been trying to free the memory but is pretty much unable to get any job done, as rows content cannot be garbage collected.

                – Julio Daniel Reyes
                Nov 14 '18 at 15:44



















              • Did you try this ? I don't think this will help, as OP is getting GC overhead limit exceeded, that means there are a lot of objects in heap

                – Schidu Luca
                Nov 14 '18 at 15:05











              • I agree that StringBuilder will help, but this will also help, because this is the cause of the GC having to run that many times. rows.add(output) -> rows.add("row1"); rows.add(output) -> rows.add("row1row2"); rows.add(output) -> rows.add("row1row2row3"); rows.add(output) -> rows.add("row1row2row3...row6422");

                – Julio Daniel Reyes
                Nov 14 '18 at 15:28













              • this does not affect in any way how many time GC runs rows.add(output), the string just keeps getting bigger, OP's mistake

                – Schidu Luca
                Nov 14 '18 at 15:37











              • The string keeps getting bigger, running out of heap memory, will cause the GC to run. GC has been trying to free the memory but is pretty much unable to get any job done, as rows content cannot be garbage collected.

                – Julio Daniel Reyes
                Nov 14 '18 at 15:44

















              Did you try this ? I don't think this will help, as OP is getting GC overhead limit exceeded, that means there are a lot of objects in heap

              – Schidu Luca
              Nov 14 '18 at 15:05





              Did you try this ? I don't think this will help, as OP is getting GC overhead limit exceeded, that means there are a lot of objects in heap

              – Schidu Luca
              Nov 14 '18 at 15:05













              I agree that StringBuilder will help, but this will also help, because this is the cause of the GC having to run that many times. rows.add(output) -> rows.add("row1"); rows.add(output) -> rows.add("row1row2"); rows.add(output) -> rows.add("row1row2row3"); rows.add(output) -> rows.add("row1row2row3...row6422");

              – Julio Daniel Reyes
              Nov 14 '18 at 15:28







              I agree that StringBuilder will help, but this will also help, because this is the cause of the GC having to run that many times. rows.add(output) -> rows.add("row1"); rows.add(output) -> rows.add("row1row2"); rows.add(output) -> rows.add("row1row2row3"); rows.add(output) -> rows.add("row1row2row3...row6422");

              – Julio Daniel Reyes
              Nov 14 '18 at 15:28















              this does not affect in any way how many time GC runs rows.add(output), the string just keeps getting bigger, OP's mistake

              – Schidu Luca
              Nov 14 '18 at 15:37





              this does not affect in any way how many time GC runs rows.add(output), the string just keeps getting bigger, OP's mistake

              – Schidu Luca
              Nov 14 '18 at 15:37













              The string keeps getting bigger, running out of heap memory, will cause the GC to run. GC has been trying to free the memory but is pretty much unable to get any job done, as rows content cannot be garbage collected.

              – Julio Daniel Reyes
              Nov 14 '18 at 15:44





              The string keeps getting bigger, running out of heap memory, will cause the GC to run. GC has been trying to free the memory but is pretty much unable to get any job done, as rows content cannot be garbage collected.

              – Julio Daniel Reyes
              Nov 14 '18 at 15:44











              0














              You have two problems in your Kotlin code:




              1. You are using strings and string concatenation - which is costly operation. You should use StringBuilder as well.

              2. You are setting output = "" outside of the foreach loop - for each iteration, you have all previous rows inside output






              share|improve this answer




























                0














                You have two problems in your Kotlin code:




                1. You are using strings and string concatenation - which is costly operation. You should use StringBuilder as well.

                2. You are setting output = "" outside of the foreach loop - for each iteration, you have all previous rows inside output






                share|improve this answer


























                  0












                  0








                  0







                  You have two problems in your Kotlin code:




                  1. You are using strings and string concatenation - which is costly operation. You should use StringBuilder as well.

                  2. You are setting output = "" outside of the foreach loop - for each iteration, you have all previous rows inside output






                  share|improve this answer













                  You have two problems in your Kotlin code:




                  1. You are using strings and string concatenation - which is costly operation. You should use StringBuilder as well.

                  2. You are setting output = "" outside of the foreach loop - for each iteration, you have all previous rows inside output







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Nov 14 '18 at 15:43









                  deydey

                  783716




                  783716






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Stack Overflow!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53302540%2fout-of-memory-error-in-kotlin-when-looping-through-a-medium-size-data-set%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      這個網誌中的熱門文章

                      Xamarin.form Move up view when keyboard appear

                      Post-Redirect-Get with Spring WebFlux and Thymeleaf

                      Anylogic : not able to use stopDelay()