![]() ![]() How could I reduce the amount of heap memory used in the process ? Is the multithread implementation with Callable correct ? How could I improve the speed of the CSV reading ? Other than that the api is running out of heap memory when running on the server, I know that a solution would be to enhance the amount of available memory but I suspect that the replace() and split() operations on strings made in the Callable(s) are responsible for consuming a large amout of heap memory. To improve speed processing, I tried to implement multithreading with Callable(s) but I am not familiar with that kind of concept, so the implementation might be wrong. The code works fine on my local machine but it is very slow : it takes about 20 seconds to process 450 columns and 40 000 lines. ![]() I am not guaranteed to have the same header between files (each file can have a completly different header than another), so I have no way to create a dedicated class which would provide mapping with the CSV headers.Ĭurrently the api controller is calling a csv service which reads the CSV data using a BufferReader. It has to read big CSV files which will contain more than 500 columns and 2.5 millions lines each. I am currently working on a spring based API which has to transform csv data and to expose them as json. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |