Unified Functional Testing Practitioners Forum

[UFT 12] Test Execution in the Cloud vs. Local VMM

Occasional Advisor

[UFT 12] Test Execution in the Cloud vs. Local VMM


Hi all,


My last role we really struggled to configure secure desktops on demand (virtual machines) for execution of both GUI & API tests.


The minimum system requirements for UFT 12.01 are the following:


Computer Processor: 1.6 Ghz or higher Operating System: Windows 7 Service Pack 1 (32-bit or 64-bit) Memory:

Minimum of 2 GB when no more than three add-ins are loaded simultaneously


  • An additional 512 MB of RAM are required when using a virtual machine.

  • Additional memory is required when loading more add-ins and when using the Save movie to results option to capture movies during run sessions.

Hard Disk Drive: 5400 RPM


My question is what the recommend VM (Virtual Machine) system footprint would be best for running both in the cloud and on local VMM (Virtual Machine Manager)?


The driving force behind this is CPU time both in the cloud and local machines costs money so getting the balance right between the minimum specification and the recommended specifications.


So far I have done some quick tests on both Cloud (Azure) and VMM (Hyper-V, VMware Workstation 10 (Win) / Parallels 9 (Mac)).


Test System Configuration(s)


Software - Clients


  • Microsoft Windows 8.1/8.0 Enterprise 
  • HP Unified Functional Testing 12.01/12.00
  • HP Service Virtualisation Designer 3.5


Software - Servers


  • Microsoft Windows 2008 R2/2012 R2 Server
  • HP Application Lifecycle Management 12.00
  • HP Service Virtualisation Server 3.5


VMM - Hyper-V Manager 6.3- Generation 2 (Dynamic Memory Support)


This came out on top for because of the dynamic memory support for both the client & server and where both assigned more memory than actually required (as ALM 12.00 requires 8GB for the installer and Quad Core CPU).


Windows: Quad Core AMD64 processor or equivalent x86-compatible processor
Linux: Quad Core AMD64 processor or equivalent x86-compatible processor
Memory (RAM)
Minimum: 8 GB
Free disk space
Minimum: 8 GB


Overall the results were extremely positive (especially from using a laptop) for both server and clients:


Server - 4.5GB (instead of the 9GB recommend)


Client - 2.8GB (which is around the 2.5GB recommend when using Virtual Machines) this also was running both HP Unified Functional Tester, Service Virtualisation Designer, ALM (HTLM 5 version not ActiveX) and Sprinter.


[IMAGE] Hyper-V Manager


NOTE: The system under test (laptop) was able to run 1 server VM (ALM/SV) and 4 client machines (4xUFTs) simulatantiously before maxing out (due to having only 8 logical processors (4 cores) on a Core i7 with 16GB of memory) after running this I decided to aim for 6GB / 2GB respectively for VMWare.


VMM - VMware Workstation 10 - Unity


Overall this was best for running 4 client (4xUFTs) simultaneously using Unity on a 4k screen meant that you could control each IDE at 1080p from the same UI or across multiple monitors (as it does not seem to scale correctly on Windows 8.1 on either a single 4k monitor or the new surface pro 2).


Because the memory footprint seemed much higher running VMWare Workstation & Unity and did manage to switch back to Hyper-V and having 12 logical processors (6 cores) on the socket 2011 it did manage to run 6 client (6xUFTs) machines simultaneously which leaves hope that the new socket 2011-3 (X99) with 16 logical processors (8 cores) should be able to manage 8-10 clients (8-10xUFTs) on a conventional desktop machine.


[IMAGE] VMWare - Unity


NOTE: The Surface Pro 2 was only tested with VMWare Workstation and not Hyper-V so was limited by memory.


VMM - Parallels Desktop 9 - Coherence


Running on the current generation Macbook Air with Windows and Mac OS combined with running two Parallels instances was noticeably slower than running them directly through BootCamp (direct access to the HAL?).


However, it was able to run both the Server & Client without any performance impact (using 8GB of LPDDR3 memory) however due to the storage requirements only two VMs instances ran in Coherence with the Apple OS.


[IMAGE] Parallels - Coherence


Cloud (Azure)


Taking what I had learnt from using standalone VMM above I spun up two cloud machines the server with 8GB of memory (quad-core) and two machines with 2GB of memory (dual-core).


As expected they both worked fine apart from having to configure the Virtual Network (VNet) endpoints to see the ALM gateway which too most of the time it seems that having ALM (SaaS) solution would have be much easier:


ALM (SaaS)


However, I did manage to get Service Virtualisation Server working too on Azure so until HP release a SaaS solution a cloud server instance will be required (which 3.6 seems to be getting there).


SV (Non-SaaS)


Standalone (Win)


Dragging out my old tablet (Iconia Tab W500) and after installing Windows 8 it seems that the minimum specifications are slightly incorrect as ran with 1GHz (Dual Core) CPU and 2GB DDR3 memory (very slowly but it ran).


NOTE: Minimum Processor is 1GHz (Dual Core)


Standalone (Mac)


This was the chance to test out the even older first generation macbook air (2008) ultrabook which was also limited to 1.6GHz (Dual Core) and 2GB DDR3 (1066MHz) which executed both UFT & SV considerably quicker than the windows tablet.


NOTE: Minimum Hard Disk Drive is 4200rpm




After experimenting with a number of different test system configurations it seems that allocating 2 logical processors and 2.5GB of dynamic memory is enough to run the client and 4 logical processors and 6GB of dynamic memory is enough to run the server.


However, in practice the footprint will increase depending on the size of test frameworks/libraries used within UFT or if you decide to run via the command prompt (In FASTMODE with a browser proxy or a real browser) and the numbers of users connecting to ALM and SV stubs/shims.


So any feedback on real-world footprints (or using other IaaS/PaaS providers such as Amazon which HP favours at the moment especially with PC/LoadRunner 12.00 load agents) would be greatly appreciated as I have a client who is looking to set up a cloud farm in the near future.


Best Regards,

Jonathon Wright
Automation Development Services | www.automation.org.uk
Part of the “Testing as a Service” community | www.Testing-Alliance.com
Registered in England Company No. 5025928

Follow my blog | www.TaaaS.net
Add me to LinkedIn | www.linkedin.com/in/automation
//Add this to "OnDomLoad" event