Yes of course. Typically the JVM requires contigious blocks of RAM, and it depends on how big the biggest contigious block of RAM is available in your machine. The maximum heap size of a Java application is limited by three factors: the process data model (32-bit or 64-bit) and the associated operating system limitations, the amount of virtual memory available on the system, and the amount of physical memory available on the system.
With other words as long as you can supply all three in the reasonable proportions, you should not experience any performance issues. Sometimes when running/executing huge scenarios from Connect IT, the system might hang for a while till it responds back and in such situations you are allowed to increase the JVM Maximum size to 512 but no more than that as it won't reduce drastically the time it takes for the scenario to get executed.