citrix – Some Downtime

The major issue with printing in Citrix boils down to the Client Side Rendering Print Provider!

Do you add printers to your remote sessions via group policy / have users add printers into their sessions using the standard windows add printer wizard? Well, if you haven’t experienced many issues yet, just you wait!

Here are some of the issues you will see that all stem from the Client Side Rendering Print Provider in RDS / Citrix (Though I can’t see why this also wouldn’t happen on a PC either).

UAC Prompt when trying to delete a session printer

Here’s a video of a standard user trying to delete a printer they added to their session.

As you can see they receive a UAC prompt! (That shouldn’t happen). So how do I resolve it?

Go to the Client Side Rendering Print Provider registry key: HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider
In the users key, expand Printers then Connections. You will need to find out the SID of the user that has the issue, there are many scripts online that can resolve a username to a SID.
Find the print connection that is having the issue, and delete the key. As you can see this key can store many connections that the user does not currently have. This can cause this part of the registry to become really large!
Restart the print spooler and try to delete the connection again.
Done!

In the video above, I don’t give the spooler enough time to restart before deleting the printer, hence why I receiver the error when trying to delete the printer the first time after I restart the spooler.

I understand you could also remove the connection from HKCUPrintersConnections, however that would just be a workaround, I’m trying to tell you why you actually get the issue in the first place!

Printer properties cannot be displayed. Operation could not be completed (error 0x00000002)

Here’s a video of the x00000002 error. Because the Client side rendering print provider caches the printers by ,,printservername,name, you can just connect to your print server using an IP or a fully qualified name and printing will work. Same printer, same print server, different results!

This is what lead to confusion for the communications staff where I work, as they thought it was an issue with the cname to the print server, because: cnameprinter didn’t work but fullyqualifiednameprintername did! This had nothing to do with the cname, and nothing to do with communications, but it just had to do with the cached entries in the Client Side Rendering Print Provider.

Case-sensitive printing

There are some applications we use internally that seem to care what case a printer is added. Think of the following scenario:

User adds a printer with lower case, and logs off server 1. The connection will be cached in the Client Side Rendering cache in lower case on server 1.
User deletes the lower case printer on server 2.
User adds the printer this time in upper case on server 1. The client side rendering key will not update to upper case, it will retain the lower case connection.

Now this is no big issue, as most programs don’t care, well… some do!

I’ve seen enough! How do I fix it?

Download and install Microsoft KB2778831

It’s not part of regular windows updates, so you have to request it (at least at the time of this writing). It resolves the issues!
Also, if you use the Citrix Printer Server / Client this issue also doesn’t appear (and I plan to move to it ASAP after testing)

The pain I experienced

It took me some time to realise that this was the issue. I turned off client side rendering in both Group Policy and on the Print Server, neither of these fixed the issue.

The communications department blamed my cname for the print server as the issue, and recommended putting the cname into the hosts file of all computers (terrible idea by the way, and I had to untangle that mess).

In the end this hotfix fixed the issues above. Install it if you want to keep your sanity!

We run Cisco Ironports to monitor internet usage and block websites to certain staff etc. When authentication is enabled with the Ironports, and you are using Internet Explorer; Internet Explorer will silently authenticate using the current users credentials against the Ironport device. The Ironport will then save the IP address of the Computer using Internet Explorer, and assume all internet requests from that IP are coming from the user that originally authenticated.

This presents a problem in Citrix XenApp, as multiple users may be using the one server, which only has one IP address. If the first user to log onto the XenApp Server is a company director with full internet access, the Ironport will grant all subsequent users that log onto the server the same internet access. Hey all the traffic is from the same IP!

To get around this issue the Ironports offer a different method than using an IP address for remembering internet access, using a session cookie. From within Identities, under Authentication in the Ironports you can create an Identity, provide a list of IP addresses (your XenApp servers) and set the surrogate type to Session Cookie. From now on, when users try to access websites from servers that have an IP in the identity created above, Internet Explorer will be given a cookie for each website the user visits. Ironports will now refer to the session cookie rather than the IP address of the server for access.

Now internet monitoring and website blocking will work for users on RDS and Citrix XenApp servers!

Well… it’s not all rainbows and unicorns…

Session cookie based authentication is… well… shithouse. It’s terrible and I have no idea why they even offer it as a solution it’s so woefully inadequate. Here are some of the issues we have encountered:

Applications that require Internet Access, and don’t know how to handle a proxy asking for authentication… don’t work
- As now the Ironports authenticate you on a site by site basis, if you are running an application that requires an external resource, the ironport will ask the application for credentials, as most applications don’t know how to handle these credential requests, they usually complain that they can’t access the internet.
- If you were using IP based authentication then once the Ironport knows you are using a certain IP, it won’t bug you for credentials for some time. This allows applications that require internet access to work!
- You could also set the websites to require no authentication to get around the issue, but do you really want to figure out the URLs of each website
Websites with urls with less than 3 characters in before the domain (such as t.co) just don’t work at all unless you want to also put them into your no authentication website list.
Websites that have dynamic content that pull from other locations will fail to work.
Sometimes websites won’t work until you have the user clear their cookies.

What has this got to do with Virtual IPs?

Well, if we could somehow give each Citrix User a unique IP, the issues above would go away! By enabling Virtual IPs on your RDS server each user will get their own unique IP address. People on Cisco forums also recommend enabling Virtual IPs to get around the issues stated above (although I seem to be unable to find official documentation on it).

So it should just be as simple as getting an IP range from your network guys and enable Virtual IPs right? Well… you would think so!

Enabling Virtual IPs

There are good resources on this already out there, I followed the following articles:

After following the above and enabling Per Session IP Virtualization, I got it working on my test sever! But then I started to do some testing, and boy… Virtual IPs break a lot of things!

IP Virtualization randomly does not work

I’m not quite sure of what causes it, but if you run the following command I have had quite a lot of success with it beginning to work again:

netsh winsock reset

IP Virtualization fails on Servers provisioned in VMWare ESX/ESXi

I did not encounter this as we are running a newer version of VMWare, but I thought I would mention the issue if you are having it. Go here for a resolution: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1038232.

Sessions with Virtual IPs cannot LDAP

This effects so many things, but mainly Outlook. Outlook won’t connect to Exchange if it can’t connect to the domain, and how does it connect to the domain? LDAP of course! You will need to install Microsoft KB2619880

Group Policy fails to apply when Virtual IPs are enabled

Since enabling Virtual IPs I have seen Group Policy fail to apply far more often. You will need to install Microsoft KB2647582

Cannot shadow users with Microsoft Remote Assistance Tool on servers with Virtual IPs enabled

I could not find anything on the internet about this, so I created a technet forum post about it here. In the end the issue is: if you don’t have an administrative user logged onto the servers console session, you will not be able to shadow users on that server.

I contacted Microsoft Support about this issue, they worked on it for a few weeks before contacting me saying this is “by design”. I have requested Microsoft update their documentation and create a KB article to reflect this issue. I have not heard back yet but I will keep you posted.

I was still having difficulty with shadowing staff, in the end I created the following registry key:

HKLM\SYSTEM\CurrentControlSet\Control\Terminal Server\TSAppSrvVirtualIPPerSessionraserver.exe

It just has to be an empty key, seemed to fix the issue for me. I found that RAServer.exe was looking for this key when I ran procmon. I created the key and the issue went away!

Conclusion

Sure you can enable Virtual IPs, just be prepared to deal with all the issues associated with it. I can also recommend that you stay away from Cisco Ironports… they are rubbish.

Category: citrix

rds and citrix printing issues