5.10 SolarisHWLog

Use this Knowledge Script to monitor hardware errors reported in Solaris logs. This script monitors logs based on regular expression patterns you can specify for the following:

hardware log name
device name
information priority
warning priority
error priority

This script is intended to monitor Solaris hardware logs. This script raises one event per log when it finds messages that match your regular expression criteria. The event detailed message orders the matched hardware messages by device name and priority.

NOTE:This Knowledge Script is available as soon as you install it. It does not require that you run the Discovery_HardwareUNIX Knowledge Script to discover resources.

5.10.1 Filtering

This Knowledge Script provides two levels of filtering for Solaris hardware messages: source and priority. Source filtering lets you select the source of the messages this script parses. Source filtering consists of the following:

The log name filter selects logs matching your UNIX-like globbing patterns
The device name filter selects device names matching your regular expression criteria
The Fault Management Daemon (FMD)-only option selects messages only from the FMD. This option requires that you configure the FMD to redirect messages to a system log.

The second type of filtering, by priority, parses and classifies hardware messages as information, warning, or error. Priority filtering consists of the following:

The error filter selects messages matching your error regular expressions. This script processes these messages as errors.
The warning filter selects messages matching your warning regular expressions. This script processes these messages as warnings.
The information filter selects messages matching your information regular expressions. This script processes these messages as informative.

At each run, this script receives hardware messages from the logs you specified. This script tests each message for source as follows:

If FMD-only is Yes, is this a FMD message?
Does the message device name match one of your device name regular expressions?

Each hardware message that passes the source tests is tested against the priority filters, in descending priority order: error, then warning, then information. The first priority match determines the message priority and removes it consideration by the remaining priority filters. Messages that do not match any priority filter are discarded.

This script groups and presents the hardware messages by priority in the event detailed message.

Selecting FMD-only Entries

If parameter Select only FMD entries is set to Yes, this script selects only those hardware messages produced by the Solaris Fault Management Daemon (FMD) from log files. All non-FMD messages are discarded.

When parameter Select only FMD entries is set to No, this script selects all hardware messages, including, if they exist, hardware messages from FMD.

Selecting a Lowest Priority

Parameter Lowest level of priority for matching sets the lowest priority this script will report in the event detailed message. For example, if you choose warn as the lowest priority, this script will only test for and report errors and warnings.

5.10.2 Script Logs for Large Message Volumes

Each event raised by this script can report up to 7000 lines of hardware messages per priority in the detailed message. If the script reports all three priorities, error, warning, and information, the event detailed message can include a maximum of 21,000 lines of hardware messages.

If the lines of hardware messages exceeds 7000 lines for a priority in an iteration, this script raises an event with the first 7000 lines in the detailed message and writes all the lines for the priority to a log it creates. The event detailed message includes the path to the log.

When this script creates a log, it writes it to this folder:

$AM_HOME/log/solaris_hw_log/

where $AM_HOME represents the AppManager home directory on the agent computer. The log name is a concatenation of the AppManager job ID, the path to the log from which the messages were parsed, and the hardware message priority:

Job<jobID>.<hardware_log_path>[-enumeration].<log_info|log_warn|log_err>

This script replaces forward slashes in the log path with underscores when it creates the log name. For example, if this script runs as AppManager job 1731, parsing more than 7000 warning and 7000 information hardware messages from log /var/adm/messages.log during an iteration, it creates one log for the information messages and one for the warning messages, naming them:

Job1731._var_adm_messages.log_info
Job1731._var_adm_messages.log_warn

If this script again receives more than 7000 hardware messages for the warning priority in a subsequent iteration, it enumerates rather than overwrites the exiting log by appending -1 to the existing log name. For example:

Job1731._var_adm_messages.log_warn
Job1731._var_adm_messages-1.log_warn

This script can create up to 12 logs for each job ID, hardware log file name, and priority. The unenumerated log always contains the most recent hardware messages and the highest enumerated log contains the oldest. If subsequent iterations again return a high hardware message volume, this script deletes the log enumerated -11, if it exists, increments the enumeration on the other logs, and writes the new hardware messages to the unenumerated log.

5.10.3 Resource Objects

Any computer running Solaris.

5.10.4 Default Schedule

The default interval for this script is Every 30 minutes.

5.10.5 Setting Parameter Values

Set the following parameters as needed:

Description	How to Set It
Monitoring Settings
File names to parse (full path, unix-like shell pattern matching notation and comma-separated)	Type a list of hardware log names for this Knowledge Script to monitor, separating the names with commas, and providing the full path. You can use UNIX-like shell pattern matching in the log names. The default is /var/adm/messages*.
Device name regex (comma-separated)	Type the device names for this Knowledge Script to monitor. Separate multiple names with commas. You can use regular expressions in the device name. The default is c[0-9]+t[0-9]+d[0-9]+[sp][0-9]+,net[0-9]+,zfs-diagnosis. NOTE: When this field is empty, this script will monitor all devices. Device name changes do not cause the job to parse any logs from the beginning even when Parse log from beginning is yes.
Info Regex (comma-separated)	Type the keywords used to find information records within hardware log files. Separate multiple keywords with commas. You can use regular expressions in the keywords. The default is notice,info,init. NOTE: When this field is empty, this script will not raise information events. Information keyword changes do not cause the job to parse any logs from the beginning even when Parse log from beginning is yes.
Warn Regex (comma-separated)	Type the keywords used to find warning records within hardware log files. Separate multiple keywords with commas. You can use regular expressions in the keywords. The default is warn,timeout,retry,degrade. NOTE: When this field is empty, this script will not raise warning events. Warning keyword changes do not cause this script to parse any logs from the beginning even when Parse log from beginning is yes.
Error Regex (comma-separated)	Type the keywords used to find error records within hardware log files. Separate multiple keywords with commas. You can use regular expressions in the keywords. The default is error,fault,fail,crit,panic,defect,major. NOTE: This script will not raise error events when this field is empty. Error keyword changes do not cause this script to parse any logs from the beginning even when Parse log from beginning is yes.
Select only FMD entries?	Select Yes to select only those messages that come from the Solaris Fault Management Daemon (FMD). Otherwise, this script selects all hardware messages including, if they exist, hardware messages from FMD.The default is unselected. NOTE:To use this option, you must configure the FMD to redirect its records to the syslog. This Knowledge Script can parse FMD records from the syslog, but has no direct interaction with the FMD.
Lowest level of priority for matching	Choose the lowest priority of information you wish to receive from the hardware log files. The choices are info, warn, and error. The default is info.
Event severity	Specify the event severity, from 1 to 40, to indicate the importance of an event when a hardware log record matches a specified priority and keyword. The default is 5.
Parse log file from beginning?	Select Yes to parse logs one time from the beginning rather than from the end. The default is unselected. By default, this script parses each log for hardware messages added since the last run. This parameter allows you to override the default behavior and parse each log one time from the beginning before resuming default behavior. This script stores parsing information internally to prevent it from repeatedly parsing a log from the beginning. This parameter affects log file parsing as follows: If yes, parse each log from the beginning one time and raise events for qualified hardware messages. In subsequent runs, parse each log file for hardware messages added since the last run. If no, store parsing information the first run but do not raise events. In subsequent runs, parse each log for hardware messages added since the last run. If changed from no to yes, parse each log from the beginning at next run. In subsequent runs, parse each log for hardware messages added since the last run. If changed from yes to no, parse each log for hardware messages added since the last run. HINT:If you want to parse each log from the beginning and anticipate a high initial log message count, NetIQ recommends you run this script with schedule run once the first time to raise events for all the hardware messages in each log. When the job completes, restart the script on a schedule to raise events for hardware messages added since the last run.
Event Settings
Raise event if specified log is missing?	Select Yes to raise an event if one of the hardware logs you specified is missing. The default is yes.
Event severity	Specify the event severity, from 1 to 40, to indicate the importance of an event when one of the hardware log files you specified is missing. The default is 5.
Raise event if AppManager fails to get metrics?	Select Yes to raise an event if AppManager cannot collect any information about the Solaris hardware logs. The default is Yes.
Event severity	Specify the event severity, from 1 to 40, to indicate the importance of an event when AppManager cannot collect any information about the Solaris hardware logs. The default is 5.
Event severity when job fails	Specify the event severity, from 1 to 40, to indicate the importance of an event when this job fails. The default is 5.
Enable degubbing?	Select Yes to enable debugging of this.