As you know, when you set up IIS for SSO you only need to enable Windows Authentication. If you enable Anonymous Authentication as well, SSO will not work.
And that's where my problem lies. Whenever we try to use https://sm.mycompany.com.au/sm/index.do, it's asking for a domain account and password (because of the IIS). If I enable Anonymous Authentication in the IIS, it will work but if you use the URL for SSO, it will now display the HP SM login page. It's either one or the other. I can't have both.
Has anyone ever done a set up where it will use Windows Authentication for anything that uses /smsso/. For those using the /sm/, it should allow for Anonymous Authentication.
Would appreciate any help/ideas/anything on this one.
Unfortunately for Windows Authentication to work with TSO via the Webtier client, you must pass a domain user. There are other products besided Windows Authentication that could be used that might get you around your issue; Siteminder or Webseal maybe. It will not work with anonymous as the user passed from IIS.
If you are willing to NOT use TSO for your non domain users, you can use the following setup for them:
You will need to setup a separate deployment of the .war/.ear file and a separate Isapi connection to make this work.