the PORTLET_MAX_ROWS_RETURNED is a hard limit for all portlets. This limit is implemented for performance reasons.
If the portlet result exceeds this number you must configure some filter and advice the users to use it or extend the parameter value in server.conf.
However, first check how mutch records the portlet statement will find. Decide if it is feasible (for performance) to extend the PORTLET_MAX_ROWS_RETURNED Or advice the users to use the search functionallity instead of portlets.
Sounds like you have the opposite problem as the norm. You have too many rows despite the config setting. I'm not sure what the answer is but if you figure out how its happening I'd love to know. I've got a special portlet where I'd like to override the config setting.
Jim is correct, Even the commented Group by clause could override PORTLET_MAX_ ROWS_RETURNED, Please add another filter and introduce the clause 'AND ROWNUM >=300' and make this filter hidden, It will solve your issue.