How to Detect “Black Hole Machines” with ControlUp in Citrix Environments

A discussion took place about the "Citrix Blackhole machine" scenario and how to detect it using ControlUp. Possible solutions were suggested, including using a combination of user sessions, CVAD Available State, and connection failure data to spot the issue and looking for Launch or LaunchOperation events on the Citrix broker for on-prem environments. It was also mentioned that Citrix Analytics may be necessary for this detection in Citrix Cloud environments. The context of this discussion is related to DaaS and ControlUp.

Read the entire ‘How to Detect "Black Hole Machines" with ControlUp in Citrix Environments’ thread below:

Hello Everyone, #, Have you come across any scenario of "Citrix Blackhole machine" and how to detect citrix blackhole machine using Control up.

I guess a combination of number of user sessions, CVAD Available State and connection failure data can help here

If we look on these data points and see a server which seems to be available, but has zero (or few) sessions and possibly connection failures, this can spot a server causing a Black Hole issue

any thoughts @member or @member?

we tried multiple paths but not successful yet, as the Citrix black hole definition is confusing from Citrix itself while they show this data in Citrix Analytics but the Maths behind is unknown. FYI – we are completely in DaaS on control panel/layer.

This has been a trending question in my organization and we have been trying to get a answer with a solution through ControlUp.

I will see if @member can come up with something creative here

Cc @member

Hey Shambhu, Trentent is out for a few days, can we connect back to you on this topic mid next week?

Sure @member

Cc : @member

Hi @member

In your environment, have you been able to find "Black hole machines" in your environment? Can you describe which machines you’ve been able to identify with this?

For example, are you purely Citrix Cloud or are you able to detect Black Hole machines purely in a Citrix Cloud VDA?

From what I’ve been able to find in my previous research, the "Black hole machines" are something Citrix detects via Citrix Cloud brokered connections and the black hole machines are detected in Citrix Analytics

Does this work for Citrix On-Prem? I’m not so sure.

I would love to validate if it’s true that Black hole machines are only detected in Citrix Cloud and Citrix Analytics is a requirement to enable this metric.

The times when a black hole machine is ‘detected’ is very vague in the Citrix definition, except for the RDS License server not having enough licenses to give out.

If Citrix is collecting the information on their Citrix Cloud broker infrastructure then ControlUp has no access to this data as it’s generated except for polling Citrix Analytics periodically for when it detects it.

For Citrix On-Prem, it may be possible to replicate this functionality by checking for Launch or LaunchOperation events on the Citrix broker:

Citrix Broker Service Events

We could create a script action that looks for these events and see how often that ran for the machine over the last 5 minutes (same as Citrix’s definition-ish) and if there are 3, 4 or 5 (Citrix has conflicting values here) detected over that time frame for a single machine then automatically set it maintenance mode.

Continue reading and comment on the thread ‘How to Detect “Black Hole Machines” with ControlUp in Citrix Environments’.  Not a member? Join Here!