Service Desk Practitioners Forum
cancel

sd_import Performance Issue

Highlighted
Kasthuri Rengan
Valued Contributor.

sd_import Performance Issue

Hi All,

I am having a sd_import performance problem.

On SD Client, I am creating a xml file thorugh sd_export. The xml file size is approximately 50MB. The time taking for importing the file in to service desk is close to 10 hours. My sytem configuration is Intel Pentium 3 Processor with 3.5GB of RAM.

Any idea as how to increase the performance while improting thorough SD Client?

Thanks in advance,
Kasthuri.
10 REPLIES
Robert S. Falko
Acclaimed Contributor.

Re: sd_import Performance Issue

Kasthuri,

Can't you import your data directly on the application server, rather than going through a client?

-Josh
Kasthuri Rengan
Valued Contributor.

Re: sd_import Performance Issue

Hi Josh,

Thanks for the reply.

The service desk server is installed on Solaris enviornment. I have created a batch file to run the data exchange through client. So only I am using the client.

Regards,
Kasthuri

Robert S. Falko
Acclaimed Contributor.

Re: sd_import Performance Issue

Kasthuri,

We use a Solaris application server, too. All of our DataExchange interfaces run directly on the server.

-Josh
Kasthuri Rengan
Valued Contributor.

Re: sd_import Performance Issue

HI Josh,

Shall a create a shell script to run the same? Will it reduce the time taken to import?

-Kasthuri
Robert S. Falko
Acclaimed Contributor.

Re: sd_import Performance Issue

Kasthuri,

I cannot know how much time you will save by eliminating the client from the processing loop. I do suggest you try it.

There are probably two other areas where you can make a significant difference in import times, depending on what you need to import.

1. If you are simply updating data, rather than creating new rows in the database, your import will probably go much faster.

2. If you can calculate a delta before doing the import, it will probably go much faster.

3. You might be able to save some more time by splitting the import job into independent segements and running each segment on a different server. This suggestion is completely theoretical. I have never tried it. However, in similar applications I have seen significant gains when the work is split amongst multiple (~3) threads.

So, what kind of data are you importing? 50MB sounds like a lot of data to import on a regular basis. If we had some more details, it might be easier to make suggestions.

-Josh
Kasthuri Rengan
Valued Contributor.

Re: sd_import Performance Issue

I am trying to run the data_exchange through server.
Gyula Matics_1
Acclaimed Contributor.

Re: sd_import Performance Issue

Please read the data excange guide on paralell import. Using several clients to import the data in parallel can significantly speed up the import process. Parallel import needs client machines and can not be run on the server itself.
JaS_4
Acclaimed Contributor.

Re: sd_import Performance Issue

Hi Kasthuri,

If you are using the sd_import from the server\bin, you can increase its java memory allocation like what we do with the server, see the sd_server.bat for example.
Note: the sd_import from the client\bin uses jview instead of java. So I don't think you can increase java memory for jview, jyst java.
It might increase the performance since you have memory to spare.
Kasthuri Rengan
Valued Contributor.

Re: sd_import Performance Issue

I have imported thorugh server and it was taking almost the same time!!

And from client, I have modified the memory allocation on the sd_import.bat to
@ECHO OFF
cd ..
set IMPORT_CMD_LINE_ARGS=
:setupArgs
if %1a==a goto doneStart
set IMPORT_CMD_LINE_ARGS=%IMPORT_CMD_LINE_ARGS% %1
shift
goto setupArgs
:doneStart
java -XX:MaxNewSize=688M -XX:NewSize=1360M -Xms1392M -Xmx1392M -cp lib\hotfix.jar;lib\servicepack.jar;lib\JClark.zip;lib\common.jar;lib\sdcommon.jar;lib\ms_interfaces.jar;lib\import.jar;lib\ui.jar com.hp.ifc.ext.imp.ImportXmlCim %IMPORT_CMD_LINE_ARGS%
cd bin

And this was also taking the same amount of time. If I try to increase the memory allocation it was telling error like
"Error occurred during initialization of VM
Could not reserve enough space for object heap
The system cannot find the path specified."

Please suggest what exactly I have to mention for the memory allocation.
Gyula Matics_1
Acclaimed Contributor.

Re: sd_import Performance Issue

Running directly on the server will not help you much.

Splitting the task into several smaller files either by running parallel import, or doing the splitting yourself will make a bigger difference.

My experience is that you can run 6-8 parallel tasks on the same server and achieve 4-5x increase in the import speed.