Troubleshooting Monthly Trigger Scheduling with ControlUp Monitor Service Time Outs

A user was having trouble scheduling a trigger to run once a month, with the script running correctly when executed manually. After checking backend data and determining a support ticket was required, the team provided a Monitor log4net file and instructions for creating the log file. It was suggested to look into the trigger timings being in different time zones, and instructions were given for setting the Monitor service time outs. In the end it was discovered that the infrastructure servers had incorrect time zones, which caused the problem. ControlUp Monitor service time out was also optimized. The Monitor service should transition from ‘started’ to ‘running’ over time. The issue is now resolved.

Read the entire ‘Troubleshooting Monthly Trigger Scheduling with ControlUp Monitor Service Time Outs’ thread below:

I’m trying to schedule a trigger to run once a month. For some reason it is not kicking off. The script runs fine when I run it manually. I’m using shared credentials for the account when running it.

checking some backend data for you

Don’t think I even see that it tried to start these, sadly I need to ask to open a support ticket. We probably need monitor logs and a test with a new scheduled trigger.

Do you create the monitor logs on your end?

or do I do it on my end?

Does the schedule look correct. I was expecting it to kick off on Saturday.

you have to add a files to the monitors to get the logs. @member can you get someone to give Glen the log4net file?

I’m also on version 8.8.0.778

yes the config looks good but I will have someone from support handle this and it looks like an issue

Thank you

Here is the Monitor log4net file and some instructions. But I do still recommend creating a support ticket so that we can assign someone to review the output.

1. Please download the logging tool
2. Save it where the cuMonitor.exe resides in (on the Monitor machine)
3. Make sure the NetworkSerivce account has Modify permissions on the ControlUp Monitor folder
4. Restart the Monitor’s service
5. A log will automatically be created in the same location with the cuMonitor.exe
6. Only after the issue was reproduced, remove the log4net file from that folder
7. Make sure that no new logs are written. If they are, restart the Monitor service
8. Please send us the logs.

Hey Glen, what makes you think the script wasn’t triggered? While scheduled triggers do not generate incident in the ControlUp incident pane. They will generate audit log entries when a script action is triggered.

If you go to Solve > audit log (under the cogwheel near the bottom left) and search for example the script name (_ExpCertReport) under the magnifying glass. Do you find anything? Assuming you set the time frame to at least the last 7 days?

@member I looked further, and I found it did run but not for all the servers in the folder. It also ran on Saturday at 2am it was schedule to run on Saturday at 12 am.

Are (some) of your monitors in a different time zone?

For example. If you schedule a trigger to run for today at 10 am. And you have monitors in 3 time zones (east, west and central US) they will run that trigger for whenever their LOCAL time is 10 am.

So your east monitor would run the trigger first. At 10 am eastern (2 pm UTC).

Then 1 hour later your central monitors would run their portion of the trigger at 10 am central (3 pm UTC).

And last but not least, 2 hours after that, your west coast monitors run the trigger at 10 am pacific (5 pm UTC)

They are all in Central time zone.

It was set for 00:00 which I’m assuming is 12am?

DMing you

check on the monitor sever for windows events 5000 & 5005 they will tell u if the automation ran sucess or fail

I’m seeing this. I have two monitors that I kick off to start but tehn go back to a stopped state.

Can you take a look in the event log on one of those monitors where they go back to Stopped and see if the issue is that the monitor service is not starting before it times out? (EventIDs 7000 or 7011)

Will check.

Also in reference to the Triggers that didn’t run the ones that ran only did off of a single monitor server.

The ControlUp Monitor service failed to start due to the following error:

The service did not respond to the start or control request in a timely fashion.

A timeout was reached (30000 milliseconds) while waiting for the ControlUp Monitor service to connect.

Can you please follow these steps:

1. Click Start, click Run, type regedit, and then click OK.
2. Locate and then click the following registry subkey:
3. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control
4. In the right pane, locate the ServicesPipeTimeout entry.
5. Right-click ServicesPipeTimeout, and then click Modify.
6. Click Decimal, type 9_0000_, and then click OK. This value represents the time in milliseconds before a service times out.
7. Restart the computer.

@member it doesnt exist. Is it a Reg_sz key?

DWORD Value

That worked for bringing the monitor back online.

should the status be showing running for all of them or started?

They should transition to ~started~ running over time

to running over time*

ok its getting there. 1 more to go.

I think I may have found the issue. the time zones are all over the place for the infra servers. not what they were supposed to be.

its working now. Thankyou all.

Continue reading and comment on the thread ‘Troubleshooting Monthly Trigger Scheduling with ControlUp Monitor Service Time Outs’.  Not a member? Join Here!