INLS283 Sample Exercise
January 17, 1999
Prof. Greg Newby
Exploring Chastity
Overview
Unix computers have a variety of tools for providing information about themselves. Some system tools may only be available to those with superuser privileges, but for the most part there are few barriers if one knows the correct commands. This exercise describes the process of exploring a system running the Silicon Graphics IRIX operating system, but should be helpful in knowing what's available from other types of systems as well.
Specific Goals
The quest for this exercise was to discover how much physical memory (RAM) Chastity has by using non-privileged programs, and to identify the maximum program size permitted (if there is a limit).
Beyond that, I wanted to find out what I could about the new scientific computing system, chastity.isis.unc.edu. I have had a username since shortly after the system was installed in summer 1998, but haven't used it much. The reason for getting an early username (the system is still in friendly testing mode) was to gain access to more memory for my research in information retrieval than is available on Ruby.
Ruby's operating system (OS) is Solaris version 2.6, which is a 32 bit OS - this translates into 2^31 bytes of memory maximum per program, or about 2 GB (gigabytes). Coincidentally, 2GB is how much physical memory, or RAM, Ruby has. The program size limit applies regardless of how much physical memory the system has. For example, statapps.unc.edu, UNC's statistical server, has more than 2GB but because it runs Solaris 2.6 only 2GB may be used by any one program. Even if the OS allowed programs to use more memory on Ruby, Ruby would need to use swap space for the extra memory, which would be slow. (Swap space is when part of a disk is used as a temporary memory extension. It's about 1000 times slower than regular physical memory. PC OSs call this virtual memory.)
The Process
Chastity is a computer made by Silicon Graphics. The login message (/etc/motd) says it was recently upgraded to 24 processors and 16GB of memory. What else could I learn about it from the Unix command line, without any special access? Here are some things I tried, and the outcome.
The first things I tried were standard commands and files to view the boot sequence. The idea was to get the messages that scroll by the system console when the machine is booted: these messages usually include things like the memory size, number of physical disks, etc.
With Linux or Solaris, the dmesg command is used to view these boot messages. Alternatively, the file /var/adm/messages contains such information (as long as the file hasn't been recycled for weekly cleaning). But there's no dmesg command for IRIX, and /var/adm/messages didn't exist. A comparable file was /var/log/SYSLOG, but it didn't contain the information I wanted even though the machine had been rebooted recently.
Two commands frequently used with Solaris to find out about the state of the computer are vmstat and iostat. vmstat (virtual machine statistics) gives an overview of how busy the system is. iostat is similar, but for the disk drives. iostat is not available with Linux, but vmstat is. Neither was found on Chastity.
After perusing files in /var/log, the 'man' command seemed the next best choice. It turned out that 'man -k' (the keyword search, also known as apropos) wasn't returning commands that are usually there. It turned out manual pages on Chastity are in /usr/share/catman, so an entry for this needed to be added to my .cshrc file's MANPATH variable so this directory was searched.
Then, commands such as these started to find some likely things to try:
One nice command is amreport. This provides statistics about downtime. Output follows:
============================ SUMMARY for chastity Count Downtime MTBI Availability
Unscheduled .................... 3 84 68252 99.96
reset action ................. 3 84 68252
Service action ................. 8 250 25594 99.88
fix/replace hardware ......... 1 38 204756
upgrade hardware ............. 2 182 102378
install patch ................ 4 26 51189
administrative: reboot ....... 1 4 204756
Total .......................... 11 334 18614 99.84
Average Uptime ................. 13628 minutes (9 days 11 hrs 8 mins)
Least Uptime ................... 0 minutes
Most Uptime .................... 54245 minutes (37 days 16 hrs 5 mins)
Average Downtime ............... 23 minutes
Least Downtime ................. 0 minutes
Most Downtime .................. 170 minutes (2 hrs 50 mins)
Logging started at ............. Fri Aug 28 16:51:12 1998
Last boot at ................... Mon Jan 11 10:48:39 1999
System has been up for ......... 9219 minutes (6 days 9 hrs 39 mins)
Thus, amreport gives us an idea about how reliable Chastity has been, and how much overall availability it provides (over 99%). top, ps and uptime (or w) can be used to find out about what's going on - who's logged in, and what processes are running.
Another good looking command is pmkstat, to provide, "high level system performance overview." However, this command said it was not licensed.
sysconf turned out to give much of the information I was seeking. This lists a whole slew of variables, a few of which are key: the number of processors, and various system limits. systune is a related command to change many limits, but it can't be run by mortals. sysconf listed many variables, including SSIZE_MAX which, I think, is the maximum amount of stack space available to a process. If not, it's perhaps something similar. The stack is where a program gets space for variables (but not necessarily where the variables are stored, which is called the heap or the data area). SSIZE_MAX was 2GB.
Chastity has many tools to evaluate its performance. This seems typical of a large mainframe- or supercomputer-class computer. There are tools to evaluate memory allocation, CPU performance, and many other things - mostly either by integrating functions into your own programs, or by running programs that watch how your program is running.
64 bits > 32 bits
What I couldn't find was a number telling me how much memory could be accessed by a process.
Some big numbers from sysconf for the 64 bit operating system on Chastity (64 bits are just out for Suns with Solaris 7, but not yet available for Linux):
LONGLONG_MAX 9223372036854775807
LONGLONG_MIN -9223372036854775808
ULONGLONG_MAX 18446744073709551615
The maximum allowed (unsigned long long) integer is about 1.84 exabytes (1.84 x 10^19). This is how many values can be stored in 2^64 (binary). The common prefixes include these (I found a listing at
http://www.twysted-pair.com/metrics.htm, and a listing of names for large numbers at http://www.optonline.com/comptons/TABLES/90000e0_T.html): KILO (thousand)
MEGA (million)
GIGA (billion)
TERA (trillion)
PETA (quadrillion)
EXA (quintrillion)
However, that doesn't mean that 1.84 exabytes can be accessed. Compare this to Ruby, which has 2GB of memory (2^31):
1024 Rubys would be 2^31 * 2^10 (2^10 is 1024) = 2^41 bytes
8196 Rubys would be 2^31 * 2^13 = 2^44
65536 Rubys would be 2^31 * 2^16 = 2^49
8589934592 Rubys (2^33) would be 2^31 * 2^33 = 2^64
In other words, 64 bit OSs allow for computers to access 2^33 or about 8.58 trillion times more memory than Ruby has. Of course, 64 bits aren't just for memory: the 64 bit limit may be applied to anything the computer keeps track of, like file sizes (number of bytes), number of files, number of users, etc.
Chastity only has 16GB (2^34, really it's more like 17.1GB), or 8 times more than Ruby.
Pay Dirt?
After lots of exploration, there was still no sign of the boot messages or of a clear way of telling how much memory was available to a process. I decided to "ls -l /var/adm" again, to look for recently updated files or directories, and noticed /var/adm/avail. Inside, there were a few filenames that reminded me of the amreport output above. One file was availlog. This file, it seems, is where system changes are tracked. By viewing it, I saw what appears to be a change history for the system. At the end of the file are entries for going from 16 to 24 CPUs, and from 8GB (8192MB) to 16GB of main memory. Aha! Here is part of my quest, although the file format is not obvious, it does correspond to the information from /etc/motd about the recent system upgrade.
But how much memory am I allowed to access? Alas, this information was not found. The /usr/include/limits.h file doesn't even list the maximum memory size. My last resort is a program I wrote last summer to try to allocate as much memory as possible, until an error is produced. It also prints out the values obtained from standard Unix C libraries for the maximum data and stack limits.
The program is 'malloctest.c' and when compiled on Chastity with 'cc -64 malloctest.c -omalloctest' yields this output:
$ ./malloctest
DATA limit is -1, max is -1
STACK limit is 67108864, max is 536870912
i is 14133755904
perror when malloc failed: : Resource temporarily unavailable
The answer, then, is that I can allocate about 14GB of space on Chastity! But this doesn't seem to be a hard limit (in fact, it was only 13.2GB a few minutes earlier). Evidently, by the non-limit for the DATA segment (-1 means no limit), there is no limit on how much memory may be allocated. Instead, a process can allocate memory until the system runs out. The STACK limit is a limitation on how many variables may be allocated, but not on their content size.
Does Chastity extend its physical memory with swap space? Evidently, not much:
$ swap -l
lswap path dev pri swaplo blocks free maxswap vswap
1 /dev/swap 0,1272 0 0 1048576 1048576 1048576 0
This swap -l output indicates there are only 1M blocks (512 bytes each) of swap space, or about 500MB. Evidently, the administrators of Chastity have taken the approach of many supercomputer administrators, and insisted that Chastity runs mostly in 'real' mode (that is, no swapping). All of the Cray supercomputers I've used enforce this same restriction.
Conclusion
Although I administered a desktop SGI workstation for years, my experience with the larger mainframe and supercomputer-class systems is limited. Because I had a specific purpose in wanting to use Chastity, it seemed prudent to get a good picture of what's available.
IRIX is very similar to Solaris, Linux and other Unix systems on the surface, but the details from an administrator's point of view are very different. Someone with more IRIX experience might have quickly uncovered the data I was seeking, or know about alternative methods for discovering what I eventually did find.
Log files (3 parts):
sci-investigation.txt
sci-investigation2.txt
sci-investigation3.txt
malloctest.c program file
Most recent update: January 17, 1999