Scheduled Jobs

We have a bunch of scheduled jobs running on various systems, mostly run by the Linux cron and a few by Quartz. There are around 25 to 30 of them. I like to categorize, so this post is about the various categories of these jobs.

Daily Emails

Data Analytics

There are reports available on the application console, but emails serve 3 purposes

1. Push paradigm - don't have to login into the console
2. Certain reports are internal, does not make sense to build an UI
3. Can forward these reports to interested parties who do not have access to the console

System Health

There is Cacti to monitor systems, but emails require no initiative from the user to see these reports, and that helps. Following information are sent out for each system (node):

1. 15 min average CPU load throughout last 24 hours
2. 15 min average memory utilization throughout last 24 hours
3. Disk usage at the end of the day

There is a helper job running every 15 minutes to collect the data from all nodes

Log Reports

1. Access log reports that provide a snapshot of number of requests to various URLs, including images and other static resources. Helps us to track hacking attempts, 404 and other errors, and general load on systems and specific services.
2. Log mining - errors and other items in application logs. We log certain messages in a parsable format, so the jobs can extract relevant data and send reports.

Email Alerts

Service Status

Say we expect remote calls (or redirects) from an external service, but haven't received any in last "t" time. A monitoring job checks the database for such events (or lack of) and sends out email alerts.

Alerts are also sent out on certain application errors, say a remote service is not accessible or on code exceptions, but these are immediate, not sent by scheduled jobs.

System Status

Sends out alerts when certain nodes are not accessible (pingable). Cacti sends email alerts on heavy load or low memory, but there is no custom job that we have written.

Data Replication Status

Sends out alerts if replication is behind schedule

Back-ups

Backs up database; this is in addition to database replication to a slave database.

Clean-ups

1. Cleans up old log files and compresses newer log files
2. Cleans up certain data from the database periodically
3. Cleans up certain files created by the operating system

Data Summarization

Data summarization for reporting purpose

Application Tasks

Recurring application tasks performing actual business processes