UCMDB and UD Practitioners Forum (Previously CMS)
cancel
Showing results for 
Search instead for 
Did you mean: 

Data Flow Probe Mysql Setup

SOLVED
Go to solution
Highlighted
ctruong
Member

Data Flow Probe Mysql Setup

Have anyone play around with the mysql setting for data flow probe? 

 

Out of the box probe are using myisam storage engine....does this engine provide better performance than innodb?

 

I've got my probe install on a 2gb box and want to see if there are way to optimized mysql performance

 

Please let me know what you think

 

 

20 REPLIES
b_bravi
Frequent Visitor

Re: Data Flow Probe Mysql Setup

From the ucmdb perspective, it is really not required to change this setting , which is out of box.

I am saying this becuase the setting which are coming along with ucmdb itself is enough .

 

Do you see some thing specific problem which is causing and hence you think of changing this ?

ctruong
Member

Re: Data Flow Probe Mysql Setup

yes as discovery run Im seeing a very slow processing time for discovered data.

UCMDB gui is almost 1-2 hour behind in term of data.  As more job being run data getting behind even further. 

 

<2012-11-29 06:30:43,704> INFO - Server Data Latest time is now: Thu Nov 29 01:59:03 EST 2012 [733607563@qtp0-695]

 

When I look at mam.autodiscovery.log it seem to me that ucmdb is constantly receiving data from probe when discovery first start and the gap in time it take for umcdb to receive result get bigger and bigger. So for some reason it take the probe longer to send result to ucmdb. So im thinking that it have something to do with the mysql database is the bottleneck. 

 

There is a known issue with MyISAM engine with the deadlock issue. Im wonderig if this is the case here. Please let me know if my theory is valid or total in the wrong direction.

 

Thank

Dima Gomel
HPE Expert

Re: Data Flow Probe Mysql Setup

Hi,

Time needed for discovery depends on many different parameters. We never find corelation between time needed for discovery and MySQL DB slowness.

There is a JMX console on the probe side. It could accessed at http://yourservername:1977.

If you go to JMX -> JobsInformation -> viewJobsStatuses, you could find avarage time needed to connect to destination along with amount of recurrence.

This will help you to find which discovery job is consuming more time.

 

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

Yes probe JMX is where I usually check for discovery status. But the problem is it job could be complete in probe jmx...but in ucmdb it still at 30% completion. And if i just leave the job active, 5 or 6 hour i would see that job status gradually go up to 100% completion.
Dima Gomel
HPE Expert

Re: Data Flow Probe Mysql Setup

As far I understadn, the slowness in processing of the populated data on the server side rather then on the probe...

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

that was my initial theory.... and maybe im understanding this the wrong way....When looking at the mam.autodiscovery.log the time it receive result from probe increase as discovery go on.
So i would see la period of 5 min where nothing is receive from the probe then that period increase to 10 then 20 etc.
Some case even after the job complete for 4 of 5 hour I see a result receive in mam.autodiscovery.log.
<2012-11-29 04:57:49,781> INFO - Received result of job progress_Task, ProbeMgr: SWVDCRVUCMDDM05, TimeStamp: 1354183062349. Time in queue: 0 [Process Results Thread-progress_Task]
<2012-11-29 04:57:49,797> INFO - finished processing result of job: progress_Task, objects to update/delete: 1/0, total time: 16, update status time: 15, ucmdb process time: 1, store results time: 0 [Process Results Thread-progress_Task]


Which is why i guess that ucmdb is requesting the result but probe's mysql is slow to response (maybe cuz of deadlock in read request)

Let me know if im looking at this wong
ctruong
Member

Re: Data Flow Probe Mysql Setup

A side question I found this value in the probe properties
# Internal: Number of connections to the Probe manager internal DataBase (MySQL DB)
appilog.agent.local.maxConnection = 20

This state that it allow maximum of 20 connection to database but when checking jmx for Probedbservice...I see that CurrentNumberOfOpenConnections is only 2 and it doesnt seem to increase. Is there another parameter that control how many connect probegw open with mysql?

thank
b_bravi
Frequent Visitor

Re: Data Flow Probe Mysql Setup

This is the maximum number of connections , but we dont use all of those.

 

As mentioned in the earlier replies, slowness is not because mysql db, there might be different problem which we are ignoring.

Dima Gomel
HPE Expert

Re: Data Flow Probe Mysql Setup

By design, UCMDB never initiate connection with the probe. It's always done from the probe side.

Probe is checking once in a while if UCMDB has discovery jobs ready for dispatching, it's also sending results back to UCMDB.

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

Ok so im off in my understanding :) thank you for the clarification

 

Where would you guy suggest I look next? what could possibly be the causing this? any test scenerios I can run?

Dima Gomel
HPE Expert

Re: Data Flow Probe Mysql Setup

In my opinion, the very first step is to understand if problem exist. :)

Check how many CIs coming to UCMDB (the throughput). Then decide what success criteria is or how many CIs UCMDB should be able to get in the certain amount of time.

 

As general direction in investigation of datain performance problems I could suggest to check reconciliation cmdb.reconciliation.audit. It will help you understand, how many CIs coming to UCMDB and how much time UCMDB spend on identification (aka reconciliation) and on the update of DB itself. 

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

My problem is somewhat like this:

 

http://support.openview.hp.com/selfsolve/document/KM978392?searchIdentifier=-640d0272%3a13b665efeb4%3a-57d0&resultType=document&documentURL=KM978392&resultsURL=%2fselfsolve%2fdocuments&allowReturn=true&searchString=slow+processing+result&searchKey=2012-12-04+05%3a26%3a44

 

Except im not getting the Not OK error. I have 10 probes and each of their queues have alot of stuff in it.

 

So my setup is as following:

 

I have 10 probe all reporting to 1 instance of ucmdb (ver 9.05 CUP8). Is there a reccommend setup that would maximize the performance of ucmdb? 

 

Is it a good idea to install these probe in a distribute setting? All 10 reporting to 1 gateway and that gateway repor to ucmdb. Is there a performance benefit to doing something like that? 

 

Any other idea setup wise to improve our discovery performance?

Dima Gomel
HPE Expert

Re: Data Flow Probe Mysql Setup

The error message regarding "RESULTS QUEUE FULL" isn't a problem, as you can see from explanation on HP SSO. 

We're processing results of 5 different probes simultaneously (1 per thread available). The meaning  that probe #6 will replace #1 when it will be done. It's a service message. 

The problem could be, for instance, in case you have unprocessed results waiting on the probe for a long time (let's say 12-24 hours). Please pay attention that first (initial) cycle is always longer. After that unchanged dicovery results won't be sent to UCMDB.

Please also notice, that if you're stopping and starting discovery job(s) every time manually, the results will always some as initial discovery cycle. The redundant filter only works for scheduled jobs.

Hope this helps.

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

What do you think about the one gateway to multiple Probe manager setup? Is that a good idea? any benefit for that type of architecture?

Thank again for the input...it been really helpful
Dima Gomel
HPE Expert

Re: Data Flow Probe Mysql Setup

It's generally not recommended unless you have security policy giving you only one open port per data center and really BIG data center. 

There is no performance benefits from this destributed probe architecture (same 5 threads on UCMDB side ;) ).

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

Well that one idea out of the window :)

Now if we can't increase the queue size...is it possible to increase the amount of data the probe send each time? If so is that recommended? or help in anyway in regard to my situation?

Dima Gomel
HPE Expert

Re: Data Flow Probe Mysql Setup

I'm sorry, I'm still not sure what problem we're trying to fix...

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

To lessen the time gap between probe actual discovery status and ucmdb GUI status.

Or are you basically saying there really nothing I can do here...I just have to let discovery run and wait for status to update?
Dima Gomel
HPE Expert
Solution

Re: Data Flow Probe Mysql Setup

Status updates are another data sent by the probe along with other data. Some gap between status on the probe and server is ok.

Please keep in mind, that new trigger CIs, coming from different discovery jobs but still not sent to the probe could keep your discovery status around 95+%. This is also intended behaviour.

 

 

Regards
-Dmitry Gomel, PMP
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at the bottom to say 'Thanks'.
ctruong
Member

Re: Data Flow Probe Mysql Setup

That is all the question I have...Greatly appreciate your input Dima

 

Have a great day 

//Add this to "OnDomLoad" event