Unix Free Tutorial

38 — UNIX System Accounting

By Scott Allen Parker

As you have learned by now, UNIX is a very complex operating system with many types of files, utilities, and programs. Your users are logging in and out, storing files, and running programs. One of the problems you may run into is keeping track of usage of the system. UNIX system accounting was created to assist you in keeping track of your users and processes. UNIX system accounting can help you troubleshoot and tune your system performance. You can even give a value to the resources on your system. This means that you can charge your users money or a fee for storing files and running processes. In this chapter, you will learn:

What is UNIX system accounting?
How do I set up and turn on the system accounting option?
How to generate report?

How Does System Accounting Work?

The moment the UNIX system is up and running, the system accounting is tracking information about the system. Information is tracked until the system shutdown. The information that is tracked is as follows:

Users logging in and out of the system
How much and many resources a user processes has taken
How much disk space has been used by the users' files

Several processes and the UNIX kernel help the system track this usage. Several of these daemons have been covered in previous chapters.

At the Start

When you boot the UNIX system into multiuser mode, UNIX runs a program called /usr/lib/acct/startup. startup is a shell script that runs other accounting programs and sets flags in the system to make the kernel and other processes to start recording information. Some of the accounting programs that are run by the startup shell script are as follows:



acctwtmp




turnacct




remove

These programs are discussed in the following sections.

acctwtmp

The /usr/lib/acct/acctwtmp program writes a record into the file called /var/adm/wtmp. wtmp is a key file of the accounting system, containing records about users connecting to the system, date changes, reboots, and system startup and shutdowns. Specifically, /var/adm/wtmp has information about the following:

A user's login name
The device the user is logging in on
The user's process id (PID)
How the user is logging in
The date and time the login was made

The record created by acctwtmp is a "boot" record containing the name of the system and the date and time the accounting system was started. You might see this information referred to as reasons in your man pages. In the startup and shutdown script, you could see:

/usr/lib/acct/acctwtmp "Accounting System ON" >> /var/adm/wtmp

/usr/lib/acct/acctwtmp "Accounting System OFF" >> /var/adm/wtmp

If you were to list the /var/adm/wtmp file you would find entries for the two examples above. The wording might be slightly different depending upon your Operating System.

turnacct

The /usr/lib/acct/turnacct program turns on the accounting system. If you look inside the startup shell script, you will see the line containing:

/usr/lib/acct/turnacct on

This program will run a special process called accton.

/usr/lib/acct/accton /var/adm/pacct

/var/adm/pacct has information about processes that are running the system. Specifically, /var/adm/pacct has information about the following:

Who is using the process
Group ID's of users using the process
The start and elapsed time of the process
The CPU timed used
The memory used
The commands run
The tty used to run or use the process.

NOTE: You will find a number of /var/adm/pacct files on your system over a period time. The reason for this is that UNIX runs a program called /usr/lib/acct/ckpacct. ckpacct will be discussed later in this chapter, but for now suffice it to say that ckpacct checks the /var/adm/pacct for its size. If the /var/adm/pacct file is more than 500 blocks, ckpacct runs turnacct to move the current pacct file to /var/adm/pacct with an incremented version number attached. For instance, /var/adm/pacct would be moved to the free name in /var/adm/pacct# (where # starts with the number 1 and is incremented by one every time an additional /var/adm/pacct is needed). The next time ckpacct runs turnacct, it will move the /var/adm/pacct file to /var/adm/pacct1, and so on. This increment insures that the /var/adm/pacct file is kept in sequence and never overwritten.

remove

/usr/lib/acct/remove will wipe out the /var/adm/acct/sum/pacct and /var/adm/acct/sum/wtmp files. The /var/adm/acct/sum directory contains accumulated summary files for most of the daily files tracked by the accounting system. You wouldn't want the file to remain between "reboots" of the accounting system or even the operating system. These files are relevant only from one boot of the accounting system to the next. We will discuss the /var/adm/acct directory later in this chapter.

Login, Run What You Will, and Logout

In a matter of minutes after the system comes up in multiuser mode, someone logs onto the system. No need to fear: the login and init programs are ready for them. login and init record the user's session by adding a record to the /var/adm/wtmp file. Next, the user runs a process, and the UNIX kernel monitors the process and writes a record about this to the /var/adm/pacct file.

There are other programs that help the accounting periodically. The /usr/lib/acct/ckpacct file, which checks /var/adm/pacct for its size, is run every hour. The ckpacct shell script runs

/usr/lib/acct/turnacct switch

to switch the current /var/adm/pacct to an archived file with a version number such as /var/adm/pacct1, /var/adm/pacct2, and so on. These archives will become important when you are recovering from a failure to process these files.

On a daily basis, the /usr/lib/acct/runacct program is run to create daily and cumulative totals for connections, fees, disk storage, and processes. You will learn more about runacct later in this chapter.

System Shutdown

When the UNIX system is shut down, the shutdown utility invokes several shell scripts found in the /sbin/rc0.d directory. One of the shells, called k22acct, runs the utility

/usr/lib/acct/shutacct

which will write a record into /var/adm/wtmp. The record is called the "reason" record. After this reason is written, the accounting system is then shutdown. Then the shutdown program finishes the system shutdown. See Chapter 34, "Starting Up and Shutting Down," for more information about the shutdown program.

Setting Up and Turning On the System Accounting Option

There are several things that you need to brush up on before starting the accounting system. The /sbin contains directories that the boot and the shutdown program use. We are concerned with only three of these directories.

/etc/rc0.d	Contains the scripts that are executed during the shutdown process
/etc/rc2.d	Contains the scripts that are executed during the boot process to multiuser mode
/sbin/init.d/acct	Contains the programs (links to shell scripts) that are executed as the UNIX system is being initialized

The /etc/rc0.d/K22acct is a shell script that shuts the accounting system down when the system is shutting down. The /etc/rc2.d/S22acct is the shell script that turns on the accounting system. Here is what you do to set up these files:

Link the /sbin/init.d/acct file to the /etc/rc0.d/K22acct.

$ link /etc/rc0.d/K22acct /sbin/init.d/acct
$

Link the /sbin/init.d/acct file to the /etc/rc2.d/S22acct.

$ link /etc/rc2.d/S22acct /sbin/init.d/acct
$

When the system is booted, the init process will run these scripts to start the system accounting option. The last thing you need to do is add entries in the crontab file. The crontab file is used by cron to run programs at predetermined times. See Chapter 20, "Scheduling Processes," for more details on cron. We need to add ckpacct, runacct, monacct, and dodisk to the crontab file to finish the accounting system setup.
Edit the crontab file to add these utilities.

$ crontab -e
Add /usr/lib/acct/ckpacct to check /var/adm/pacct every hour to archive the pacct file after its size is more than 500 blocks.

0 * * * * /usr/lib/acct/ckpacct
Add /usr/lib/acct/runacct to run daily to process the accounting files to prepare daily and cumulative summary files. It is recommended that you run this file at off-hours of the morning. You can pick any time. For this example, we will use 1:30 a.m.

30 1 * * * /usr/lib/acct/runacct 2> /var/adm/acct/nite/fd2log

/var/adm/acct/nite/fd2log is a log file that you look at to verify that runacct is running cleanly.
Add /usr/lib/acct/monacct to run monthly. The monacct file takes data stored in the /var/adm/acct/sum directory and creates a monthly report of all daily totals.

30 3 * 1 * /usr/lib/acct/monacct
Add /usr/lib/acct/dodisk program to do disk usage accounting. It is recommended that you run this program once a week and before runacct is executed daily.

00 22 * * 4 /usr/lib/acct/dodisk
Shutdown and reboot your system to activate the accounting system.

The Accounting System Programs

Remember the processes that you add to the crontab file. Those processes are essential to keep track of your system usage.

runacct

/usr/lib/acct/runacct is a shell program that is executed every day to process system usage. It will create daily summary files for the /usr/lib/acct/prdaily and /usr/lib/acct/monacct programs. prdaily is run by runacct to write daily accounting information to the /var/adm/acct/sum/rprtMMDD file. MMDD is the month and day the file was created. monacct is the month usage report, which will be covered later in this chapter. There can be one of these files for every day of the week. runacct actually writes information to several files.

/var/adm/pacct?	Contains process information. ? represents the incremented /var/adm/pacct file.
/var/adm/wtmp	Contains user information
/var/adm/fee	Contains fees accessed for usage
/var/adm/acct/nite/disktacct	Contains the disk space usage

You can find the output of the runacct program in the /var/adm/acct/nite directory. Other files in the /var/adm/acct/nite directory are as follows:

lock and lock1	These files may or may not exist. If they do exist, runacct will not run. It will "think" that it is already running. If you get an error concerning these files during an attempted execute of runacct, remove them with rm (remove command).
lastdate	This file records the last date that runacct was executed. This file is checked to prevent runacct from being executed more than once daily.
fd2log	This file contains the message generated by runacct. It will contain important error information in case runacct fails to run.

NOTE: If runacct does have an error, root will be notified by mail. It will write information to /var/adm/acct/nite/fd2log and remove the lock files.

dodisk

The /usr/lib/acct/dodisk shell script cumulates disk usage information. This shell script program runs three programs.

diskusg	Collects file data by reading the file INODES
acctdusg	Collects file statistics in the file system
acctdisk	Formats the data from diskusg or acctdusg

NOTE: Only one of the file data accounting programs needs to run. /usr/lib/acct/diskusg and /usr/lib/acct/acctdusg output the same information, but how they approach the information differs. diskusg is much faster than acctdusg because it looks at the lowest level of file information in the INODE. To toggle between the two, the dodisk can invoke the -o option. The following script:

/usr/lib/acct/dodisk /dev/dsk/c1t0d0s2

will run the diskusg method against the device file name of /dev/dsk/c1t0d0s2. If the device name is not specified, then diskusg will look in the /etc/vfstab file and process all the devices. This is very similar to the fsck command that looks at the file system's INODE's when it checks the file system at boot time. This is much faster. The following script:

/usr/lib/acct/dodisk -o /user

will run the acctdusg method against the /user file system mounting point. If the mount point is not specified, the root mounting point is used.

Remember, if you want to use acctdusg, add the -o option to the dodisk line in the crontab file.

acctdisk will write the formatted output to the /var/adm/acct/nite/disktacct file. This file will have the following information about users' files on the system:

The user's login name
The user's id number
The number of blocks in use in the user's files

WARNING: dodisk stores all this information in /var/adm/acct/nite/disktacct. Each and every time dodisk is executed, it overwrites the /var/adm/acct/nite/disktacct file. Executing dodisk more than once daily should be avoided.

chargefee

If you are in a Computer Services department or part of a service provider, you may elect to charge other departments or users for the resource they use. UNIX has provided a program called chargefee that will charge your user for a number of services. The charges that are generated by chargefee are stored in /var/adm/fee. Say that carolynp sends me a message to mount a tape for her on my system and I charge $1.50 for every mount.

$ chargefee carolynp 1.50

$

An entry in /var/adm/fee would be made having carolynp, her user id number, and 1.50. Later in my monthly accounting report charges for mounting tapes, restoring files, etc. can be polled into an invoice billed to the user. Most places will normally charge for processor time and disk space on a monthly basis. The monacct program, which you can read about next, will generate a nice report to run charge-back scripts against the invoice users.

monacct

monacct runs monthly, or you can run it whenever your fiscal period ends, to generate files that summarize the statistic files created by dodisk and runacct. These files are stored in the /var/adm/acct/fiscal directory. After the monacct program is run, the files created by dodisk and runacct removed and reset for the next fiscal period.

acctcom

The acctcom utility allows you to see the accounting system at any given time. You can execute this command from the command line with several different options.

$ acctmon -a

This will show the average statistics about processes.

$ acctmon -r

This will show the amount of user time per total time (system time plus user time).

$ acctmon -u zachp

This will show all the processes belonging to the user zachp.

$ acctmon -O 20

This will show all the processes running longer than 20 seconds.

To see more options for the acctcom command, please refer to your man pages. acctcom will look in the /var/adm/pacct? files for these little records.

Daily Reports

runacct generates a number of reports.

The Daily	Shows the usage of ports on your system.
The Daily Usage	Shows the system resource used by your users during the daily period.
The Daily Command Summary	Shows the commands run on your system and resources those commands used. This report can be essential in helping you determine the process that might bottleneck your system.
The Last Login	Tells you the last time a login id was used by a user. This report can help you remove unused login id's and directories associated with those id's.

Daily Report

The Daily Report can be found in the /var/adm/acct/nite/lineuse file.

$ cat /var/adm/acct/nite/lineuse

Apr 06 01:33 1994  DAILY REPORT FOR excelsior Page 1

from Tue Apr 05 05:10:41 1994

to   Wed Apr 06 01:31:20 1994

1        runacct

1        accton

TOTAL DURATION IS 5155 MINUTES

LINE     MINUTES   PERCENT  # SESS   # ON     #OFF

ttyp01   1541      30       4        9        5

ttyp10   2564      50       25       8        6

ttyp13   1050      20       15       3        4

TOTALS   5155      100      44       20       10

$

The detail of this report column by column are as follows:

LINE	The port that was accessing the system.
MINUTES	The number of minutes the line was in usage during the daily period.
PERCENT	The number of minutes in use divided by TOTAL DURATION. TOTAL DURATION is the number of minutes the system was in multiuser mode.
# SESS	The number of times the port was accessed to log in to the system.
# ON	The number of times the port was used to log in the user into the system. Hey, if you see that the # SESS is very large compared to the # ON, then you have a problem. There might be someone hacking your system on that port.
# OFF	The number of logoffs that occurred at that port and the number of interrupts like Ctrl-c, EOF, etc.

Daily Usage Report

The Daily Usage Report can be found in the /var/adm/acct/nite/daytacct file.

$ cat /var/adm/acct/nite/daytacct

Apr 06 01:33 1994  DAILY USAGE REPORT FOR excelsior Page 1

     LOGIN   CPU (MINS)  KCORE-MINS   CONNECT (MINS)  DISK    # OF   # OF  # DISK  FEE

UID  NAME   PRIME NPRIME PRIME NPRIME  PRIME  NPRIME  BLOCKS  PROCS  SESS  SAMPLES

0    TOTAL   6     13     7     14      165      67      0     1020     6      0    0

0    root    3      7     1      8        0       0      0      400     0      0    0

3    sys     0      3     0      1        0       0      0       51     0      0    0

4    adm     0      1     0      1        0       0      0      251     0      0    0

5    uucp    0      0     0      0        0       0      0       60     0      0    0

1091 carolyn 2      1     4      3      140      47      0      249     2      0    0

2155 zach    1      1     2      1       25      20      0        9     4      0    0

$

Here it is column by column:

UID	The user's identification number.
LOGIN NAME	The user's name.
CPU (MINS)	The amount of time the user's program required the use of CPU. This is rounded up to the nearest minute.
KCORE-MINS	The amount of memory per minute used to run the programs. This is rounded up to the nearest kilobyte.
CONNECT (MINS)	Total time the user was actually connected to the system.
DISK BLOCKS	The number of disk blocks used. This sum is placed by dodisk.
# OF PROCS	The number of processes the user executed.
# OF SESS	The number of sessions the user incurred by logging in to the system.
# DISK SAMPLES	The number of times acctdusg or diskusg was run to cumulate the average number of DISK BLOCKS.
FEE	The total amount of usage charges accessed to the user for this given period.

NOTE: You might have noticed that I didn't mention PRIME and NPRIME in the above list. PRIME is the prime-time hours for processing, and NPRIME is the non-prime hours for processing. For instance, holidays would not be considered prime-time hours. You would expect that a majority of your users would not be on the system during the holiday. The file /etc/acct/holidays allows you to tailor the non-prime times for your company. Why would this be important? I want to bill my customer a premium rate for using my system during the days or during the heavy processing hours. I will charge a lower rate at non-prime hours. For example, my prime-time hours are from 8:00 a.m. (800 hours) to 6:30 p.m. (1830 hours) for 1994. I would add the following entry in the /etc/acct/holidays file.

# Prime Time Hours for 1994
1994 0800 1830

Here is a sampling of my /etc/acct/holidays file:

$ cat /etc/acct/holidays
#
# Holidays
#
0101 New Year's Day
0528 Memorial Day
0704 Independence Day
#
# Prime Time Hours for 1994
#
1994 0800 1830
$

Daily Command Summary Report and Total Command Summary Report

The Daily Command Summary Report can be found in the /var/adm/acct/nite/daycms file.

$ cat /var/adm/acct/nite/daycms

Apr 06 01:32 1994  DAILY COMMAND SUMMARY REPORT FOR excelsior Page 1

                                     TOTAL COMMAND SUMMARY

COMMAND NUMBER    TOTAL  TOTAL    TOTAL   MEAN   MEAN     HOG    CHARS  BLOCKS

NAME     CMDS  KCOREMIN CPU-MIN REAL-MIN SIZE-K CPU-MIN  FACTOR  TRNSFD   READ

TOTALS   2050      3.57   21.59   157.57   0.21    0.02   0.14  6570519   2726

csh       171      2.50    2.56    10.71   0.45    0.02   0.05   257429    212

grep       14      0.10     .56     2.71   0.40    0.01   0.34    17537     42

more        5      0.04    0.09     1.01   0.59    0.01   0.45    25414      2

awk         2      0.01    0.12     1.71   0.15    0.01   0.55      529      5

  .

  .

  .

$

The Total Command Summary Report looks like the preceding report with one exception. It is a monthly summary showing total accumulated since last month or execution of monacct. This report can be seen in the /var/adm/acct/sum/cms file. Here are the column-by-column details.

COMMAND NAME	The name of the command.
NUMBER COMMANDS	The total number of times the command has been executed.
KCOREMIN	The total cumulative kilobytes segments used by the command.
TOTAL CPU-MIN	The total processing time in minutes.
REAL-MIN	The actual processing time in minutes.
MEAN SIZE-K	The mean of TOTAL KCOREMIN divided by execution.
MENU CPU-MIN	The mean of executions divided by total processing time in minutes.
HOG FACTOR	The total processing time divided by elapsed time. This is the utilization ratio of the system.
CHARS TRNSFD	The total number of reads and writes to the file system.
BLOCKS READ	The total number of physical block reads and writes.

NOTE: For purposes of illustration, I have deleted the PRIME and NPRIME column from this report. On your system, these will be there for you to view. See the previous note box about what PRIME and NPRIME represent.

Last Login Report

The Last Login Report can be found in the /var/adm/acct/sum/loginlog file. This report has the last login that your users have made on your system. Any entry that you find that is several months old could be a candidate to purge from your system.

$ cat /var/adm/acct/nite/daycms

Apr 06 01:32 1994  LAST LOGIN Page 1

   ...

93-01-05 briano    94-01-11 philp     94-02-21 deanm     94-03-01 stacyh    

93-01-13 jordang   94-01-11 kittyw    94-02-21 richards  94-03-01 zachp     

93-10-03 bradj     94-01-11 cindym    94-02-21 davidb    94-03-01 jimg      

93-10-07 deborahf  94-01-11 franh     94-02-21 seanm     94-03-11 mitzig    

93-11-05 gaylej    94-01-21 gregc     94-02-21 maryi     94-03-12 chrisd    

93-12-05 keithd    94-01-21 wayneb    94-02-24 kristih   94-03-17 lynetteq  

93-12-11 markt     94-01-21 matthewu  94-02-24 sandrad   94-03-20 sharonc   

93-12-13 robh      94-01-21 philk     94-02-24 gregb     94-03-21 margaret  

93-12-25 cindyk    94-01-21 dianah    94-02-24 daniels   94-03-21 paulas    

94-01-05 deniseo   94-01-21 richc     94-02-24 lauric    94-03-22 mikes     

94-01-05 gingera   94-02-05 carolynp  94-02-24 keitho    94-03-25 scottp    

94-01-05 greggb    94-02-13 jimg      94-02-24 joew      94-04-01 kathye    

94-01-05 katyo     94-02-15 matthewh  94-02-24 virgilp   94-04-05 daveh     

94-01-05 viginiap  94-02-15 douga     94-03-01 briant    94-04-08 stepht    

94-01-05 mollyp    94-02-15 cameront  94-03-01 sneakerp  94-04-10 sugerp    

94-01-05 bwhitmer  94-02-17 beths     94-03-01 carola    94-04-11 rosemari  

   .

   .

   .

$

Summary

In this chapter, you learned how to set up the accounting system to track your users and the processes they run. UNIX System Accounting can be a useful tool to help you tune your system and to plan for future expansion of hard disks, memory, and processors. This is the most common usage of the accounting system. If you are a provider for UNIX resource, such as connections to the Internet, the accounting system allows you to bill those users for the use of your system.