Oracle7 Parallel Server Concepts and Administrator's Guide Go to Product Documentation Library
Library
Go to books for this product
Product
Go to Contents for this book
Contents
Go to Index
Index



Go to previous file in sequence Go to next file in sequence

Administering Multiple Instances


Justice is a machine that, when someone has once given it the starting push, rolls on of itself.

John Galsworthy: Justice. Act II.

This chapter describes how to administer instances of a parallel server. It includes the following topics:


Overview

Set up and then start up instances for a parallel server using the following general procedure:

To Define and Start Instances

See Also: "Starting Up and Shutting Down" in Oracle7 Server Administrator's Guide.


Defining Multiple Instances with Parameter Files

When an instance starts up, Oracle uses the values found in an initialization parameter file to create the System Global Area (SGA) for that instance. You can use various approaches to define multiple instances:

Using a Common Parameter File for Multiple Instances

A common parameter file for all instances, shown in Figure 19 - 1, can make administration easy. If file systems are shared among nodes, you can update all instances by making a change in only one place.

Figure 19 - 1. Instances with a Common Parameter File

Most clustering systems, however, do not share file systems. In this case you would have to make for each node a separate physical copy of the common file.

Using Individual Parameter Files for Multiple Instances

Individual parameter files are useful when many parameters should differ from instance to instance. For example, initialization parameters to create difference size SGAs for different size machines may improve performance dramatically.

Figure 19 - 2. Instances with Individual Parameter Files

Embedding a Parameter File Using IFILE

By setting the IFILE parameter, each individual parameter file can embed an additional parameter file containing common values. This approach is illustrated in Figure 19 - 3.

Figure 19 - 3. Instances with Individual Parameter Files and IFILE

In a parallel server, some initialization parameters must have the same values for every instance, whether individual or common parameter files are used. By referencing the same file using the IFILE parameter, instances which have individual parameter files can ensure that they have the correct parameter values for those which must be identical, while allowing individual values for parameters which can differ.

Instances must use individual parameter files in the following cases:

Example

For example, a Server Manager session on the local node can start up two instances on remote nodes using individual parameter files named INIT_A.ORA and INIT_B.ORA:

SET INSTANCE instance1; 
STARTUP PFILE=init_a.ora PARALLEL; 
SET INSTANCE instance2; 
STARTUP PFILE=init_b.ora PARALLEL; 

Here, "instance1" and "instance2" are SQL*Net aliases for the two respective instances, as defined in TNSNAMES.ORA.

Both individual parameter files can use the IFILE parameter to include parameter values from the file INIT_COMMON.ORA. They can reference this file as follows:

INIT_A.ORA:

IFILE=INIT_COMMON.ORA
INSTANCE_NUMBER=1
THREAD=1

INIT_B.ORA:

IFILE=INIT_COMMON.ORA
INSTANCE_NUMBER=2
THREAD=2

The INIT_COMMON.ORA file can contain the following parameter settings, which must be identical on both instances.

DB_NAME=DB1
CONTROL_FILES=(CTRL_1,CTRL_2,CTRL_3)
GC_FILES_TO_LOCKS="1=600:2-4,9=500EACH:5-8=800"
GC_ROLLBACK_SEGMENTS=10
GC_SEGMENTS=10
LOG_ARCHIVE_START=TRUE

Each parameter file must contain the same values for the CONTROL_FILES parameter, for example, because all instances share the control files.

To change the value of a common initialization parameter, you would only have to modify the file INIT_COMMON.ORA, rather than changing both individual parameter files.

IFILE Usage

When you specify parameters which have identical values in a common parameter file referred to by the IFILE parameter, you can omit parameters for which you are using the default values.

If you use multiple Server Manager sessions on separate nodes to start up the instances, each node must have its own copy of the common parameter file (unless the file systems are shared).

If a parameter is duplicated in an instance-specific file and the common file, or within one file, the last value specified overrides earlier values. You can therefore ensure the use of common parameter values by placing the IFILE parameter at the end of an individual parameter file. Placing IFILE at the beginning of the individual file allows you to override the common values.

You can specify IFILE more than once in a parameter file to include multiple common parameter files. Unlike the other initialization parameters, IFILE does not override previous values. For example, an individual parameter file might include a file INIT_COMMON.ORA and separate command files for the LOG_* and GC_* parameters:

IFILE=INIT_COMMON.ORA
IFILE=INIT_LOG.ORA
IFILE=INIT_GC.ORA
LOG_ARCHIVE_START=FALSE
THREAD=3
ROLLBACK_SEGMENTS=(RB_C1,RB_C2,RB_C3)

The individual value of LOG_ARCHIVE_START overrides the value specified in INIT_LOG.ORA, because the IFILE = INIT_LOG.ORA appears before LOG_ARCHIVE_START parameter specification. The individual GC_* values specified in INIT_GC.ORA override any values specified in INIT_COMMON.ORA, because IFILE = INIT_GC.ORA comes after IFILE = INIT_COMMON.ORA.

See Also: "Instance Numbers and Startup Sequence" [*].

"Threads of Redo" [*].

"Parameters Which Must Be Identical" [*].

Specifying a Non-default Parameter File with PFILE

The PFILE option of the STARTUP command allows you to specify a parameter file other than the default file when you start up an instance. The parameter file specified by PFILE must be on a disk accessible to the local node, even for an instance on a remote node.


Setting Initialization Parameters for the Parallel Server

This section discusses initialization parameters which are important for a parallel server.

See Also: "Initialization Parameters" [*] for a complete list of parallel server parameters.

Oracle7 Server Reference for details about all other Oracle initialization parameters.

GC_* Global Constant Parameters

Initialization parameters with the prefix GC (Global Constant) are relevant only for a parallel server. The settings of global constant parameters determine the size of the collection of global locks which protect the database buffers on all instances. The settings you choose affect the use of certain operating system resources.

The first instance to start up in shared mode determines the values of the global constant parameters for all instances. The control file records the values of the GC_* parameters when the first instance starts up.

When another instance attempts to start up in shared mode, Oracle compares the values of the global constant parameters in its parameter file with those already in use and issues a message if any values are incompatible. The instance cannot mount the database unless it has the correct values for its global constant parameters.

The global constant parameters for a parallel server are:

GC_DB_LOCKS GC_ROLLBACK_LOCKS
GC_FILES_TO_LOCKS GC_ROLLBACK_SEGMENTS
GC_FREELIST_GROUPS GC_SAVE_ROLLBACK_LOCKS
GC_LCK_PROCS GC_SEGMENTS
GC_RELEASABLE_LOCKS GC_TABLESPACES
See Also: "Allocating PCM Instance Locks" [*] for details on setting these parameters.

Parameter Notes for Multiple Instances

Multi-instance issues concerning initialization parameters are summarized in the following table.

Parameter Parallel Server Notes
CHECKPOINT_PROCESS In Oracle Parallel Server your database may have more datafiles. To speed up checkpoints, enable the CHECKPOINT_PROCESS parameter.
DELAYED_LOGGING_BLOCK_ CLEANOUTS If set to True, this parameter could potentially reduce pinging between instances.
DML_LOCKS This parameter must be identical on all instances only if set to zero.
INSTANCE_NUMBER If specified, this parameter must have unique values for different instances.
LOG_ARCHIVE_FORMAT You must include thread number.
MAX_COMMIT_PROPAGATION_ DELAY If you want commits to be seen immediately on remote instances, you may need to change the value of this parameter.
NLS_* parameters This parameter can have different values for different instances.
PROCESSES This parameter must have a value large enough to allow for all background processes and all user processes in an instance. Some operating systems can have additional DBWR processes. Defaults for the SESSIONS and TRANSACTIONS parameters are derived directly or indirectly from the value of the PROCESSES parameter If you do not use the defaults, you may want to increase some of these parameter values to allow for LCKn and other optional background processes.
RECOVERY_PARALLELISM To speed up the roll forward or cache recovery phase, you may want to set this parameter.
ROLLBACK_SEGMENTS Specify the private rollback segments for each instance.
THREAD If specified, this parameter must have unique values for different instances.
Table 19 - 1. Initialization Parameter Notes for Multiple Instances

See Also: Oracle7 Server Reference for details about each parameter.

Parameters Which Must Be Identical on Multiple Instances

Certain initialization parameters that are critical at database creation or that affect certain database operations must have the same value for every instance in a parallel server. For example, the values of DB_BLOCK_SIZE and CONTROL_FILES must be identical for every instance. Other parameters can have different values for different instances; for example, INIT_SQL_FILES can have any value because it is ignored except when the database is created.

The following initialization parameters must have identical values for every instance in a parallel server:

CACHE_SIZE_THRESHOLD
CONTROL_FILES
CPU_COUNT
DB_BLOCK_SIZE
DB_FILES
DB_NAME
DELAYED_LOGGING_BLOCK_CLEANOUTS
DML_LOCKS (must be identical only if set to zero)
GC_DB_LOCKS
GC_FILES_TO_LOCKS
GC_FREELIST_GROUPS
GC_LCK_PROCS
GC_ROLLBACK_LOCKS
GC_ROLLBACK_SEGMENTS
GC_SAVE_ROLLBACK_LOCKS
GC_SEGMENTS
GC_TABLESPACES
LOG_FILES
MAX_COMMIT_PROPAGATION_DELAY
PARALLEL_DEFAULT_MAX_INSTANCES
PARALLEL_DEFAULT_MAX_SCANS
PARALLEL_DEFAULT_SCANSIZE
ROLLBACK_SEGMENTS
ROW_LOCKING
SERIALIZABLE


Setting DLM Parameters

Depending on the implementation of your platform's distributed lock manager, you may need to manually set DLM initialization parameters. In particular, you may need to set parameters relating to:

To set appropriate values for these DLM parameters, you need to first determine the number of PCM and non-PCM locks your system will require.

Refer to your operating system documentation for complete details on DLM initialization parameters.

See Also: "Allocating PCM Locks" [*].

"Adjusting DLM Capacity for Non-PCM Locks" [*].


Creating Database Objects for Multiple Instances

Creating a database automatically starts up a single instance in exclusive mode. Before you can start up multiple instances, however, you must perform certain administrative operations. These tasks may include:

You can perform these operations with a single instance in either exclusive or shared mode.

See Also: "Creating Additional Rollback Segments" [*].

"Threads of Redo" [*].

"What is the Total Number of PCM Locks & Resources Needed?" [*].


Starting Up Instances

An Oracle instance can start up in either exclusive mode or shared mode. This section includes the following topics:

To Start an Instance Using SQL

	CONNECT username/password AS SYSDBA

	STARTUP NOMOUNT

	ALTER DATABASE database_name MOUNT [EXCLUSIVE | PARALLEL]

	ALTER DATABASE OPEN 

The Server Manager command STARTUP with the OPEN option performs steps 3, 4, and 5.

To Start an Instance Using Server Manager

	STARTUP OPEN database_name [EXCLUSIVE | PARALLEL]

Starting up in Exclusive Mode

Exclusive mode is required whenever you change the archiving mode (ARCHIVELOG or NOARCHIVELOG). To change the archiving mode, the database must be mounted but not open.

If an instance mounts a database with the EXCLUSIVE option, no other instance can mount the database.

Before you can start up an instance in exclusive mode, you must shut down all instances running in shared mode. A single instance running in shared mode is not the same as an instance running in exclusive mode, and the last instance running in shared mode does not automatically revert to exclusive mode.

An instance starting up in exclusive mode can specify an instance number with the INSTANCE_NUMBER parameter. This is only necessary if the instance will perform inserts and updates and if the tables in your database use the FREELIST GROUPS storage option to allocate free space to instances. If you start up an instance just to perform administrative operations in exclusive mode, you can omit the INSTANCE_NUMBER parameter from the parameter file.

An instance starting up in exclusive mode can also specify a thread other than 1, to use the online redo log files associated with that thread.

See Also: "Using Free List Groups to Partition Data" [*] for more information.

Starting Up in Shared Mode

In a parallel server, each instance must mount the database in shared mode. Each initialization parameter file for each instance must have the SINGLE_PROCESS parameter set to FALSE. Before you start up multiple instances in shared mode, you must create at least one rollback segment for each instance sharing the same database and enable a thread containing at least two groups of redo log files for each additional instance.

If one instance mounts a database with the PARALLEL option, other instances can also mount the database with the PARALLEL option but not with the EXCLUSIVE option.

If you omit the PARALLEL option, the instance tries to start up in exclusive mode by default.

There is no difference between the options PARALLEL and SHARED in either the ALTER DATABASE statement or the STARTUP command.

Retrying to Mount a Database in Shared Mode

If you attempt to start an instance and mount a database in shared mode while another instance is currently recovering the same database, your new instance cannot mount the database until the recovery is complete.

Rather than repeatedly attempting to start the instance, you can use the RETRY option in the STARTUP PARALLEL command or in the dialog box. This causes the new instance to retry every five seconds to mount the database until it succeeds or has reached the retry limit. For example:

STARTUP OPEN maildb PARALLEL RETRY

To set the maximum number of times the instance attempts to mount the database, use the Server Manager SET command with the RETRY option; you can specify either an integer (such as 10) or the keyword INFINITE.

If the database cannot be opened for some reason other than recovery by another instance, then the RETRY will not repeat. For example, if the database was mounted EXCLUSIVE by one instance, then trying the STARTUP PARALLEL RETRY command will not work for another instance.

Instance Numbers and Startup Sequence

When an instance starts up, it acquires an instance number which maps the instance to one group of free lists for each table created with the FREELIST GROUPS storage option.

An instance can specify its instance number explicitly by using the initialization parameter INSTANCE_NUMBER when it starts up in either shared or exclusive mode. If an instance does not specify the INSTANCE_NUMBER parameter, it automatically acquires the lowest available number.

Startup order determines the instance numbers for instances which do not specify the INSTANCE_NUMBER parameter. Startup numbers are difficult to control if instances start up in parallel, and they can change after instances shut down and restart.

Instances which use the INSTANCE_NUMBER parameter must specify different numbers. The Server Manager command SHOW PARAMETERS INSTANCE_NUMBER can show the current instance number each instance is using. This command displays a null value if an instance number was assigned based on startup order.

After an instance shuts down, its instance number becomes available again. If a second instance starts up before the first instance restarts, the second instance can acquire the instance number previously used by the first instance.

Instance numbers based on startup order are independent of instance numbers specified with the INSTANCE_NUMBER parameter. After an instance acquires an instance number by one of these methods (either with or without INSTANCE_NUMBER), another instance cannot acquire the same number by the other method. All numbers are unique, regardless of the method by which they are acquired.

Always use the INSTANCE_NUMBER parameter if you need a consistent allocation of extents to instances for inserts and updates. This allows you to maintain data partitioning among instances.

See Also: "Rollback Segments" on page 6 - 7.

"Creating Additional Rollback Segments" [*].

"Threads of Redo" [*].

"Using Free List Groups to Partition Data" [*] for information about allocating free space for inserts and updates.


Specifying Instances

When performing administrative operations in a multi-instance environment, you must be sure that you have specified the correct instance. This section includes the following topics:

Differentiating Between Current and Default Instance

Some Server Manager commands apply to the instance to which Server Manager is currently connected, and others apply to the default instance.

default instance

The default instance is on the machine where you initiate Server Manager. Server Manager commands which cannot be used while you are connected to an instance (such as executing a host command) apply to the default instance.

current instance

The current instance is determined by the CONNECT command. Server Manager commands which can be used while you are connected to an instance apply to the current instance.

The current instance can be different from the default instance if you specify a connect string in the CONNECT command.

SQL*Net must be installed to use the SET INSTANCE or CONNECT command for an instance running on a remote node.

See Also: Your platform-specific Oracle documentation, for more information about installing SQL*Net and the exact format required for the connect string used in the SET INSTANCE and CONNECT commands.

How SQL Statements Apply to Instances

Instance-specific SQL statements apply to the current instance. For example, the statement ALTER DATABASE ADD LOGFILE only applies to the instance to which you are currently connected, rather than the default instance or all instances.

ALTER SYSTEM CHECKPOINT LOCAL applies to the current instance. By contrast, ALTER SYSTEM CHECKPOINT or ALTER SYSTEM CHECKPOINT GLOBAL applies to all instances.

ALTER SYSTEM SWITCH LOGFILE applies only to the current instance. To force a global log switch, you can use ALTER SYSTEM ARCHIVE LOG CURRENT. The THREAD option of ALTER SYSTEM ARCHIVE LOG allows you to archive online redo log files for a specific instance.

How Server Manager Commands Apply to Instances

When you initiate Server Manager, the commands you enter are relevant to the default instance, which is also the current instance.

This is true until you use the SET INSTANCE command to set the current instance. From that point onwards, all Server Manager commands operate on the current instance.

Server Manager Command Associated Instance
ARCHIVE LOG always applies to the current instance
CONNECT uses the default instance if no instance is specified in the CONNECT command
CONNECT INTERNAL always applies to the current instance. A privileged Server Manager command.
HOST applies to the node running the Server Manager session, regardless of the location of the current and default instances
MONITOR MONITOR display screens identify the current instance, not the default instance, in the upper left corner.
RECOVER does not apply to any particular instance, but rather to the database
SHOW INSTANCE displays information about the default instance, which can be different from the current instance
SHOW PARAMETERS displays information about the current instance
SHOW SGA displays information about the current instance
SHUTDOWN always applies to the current instance. A privileged Server Manager command.
STARTUP always applies to the current instance. A privileged Server Manager command.
Table 19 - 2. How Server Manager Commands Apply to Instances

Additional Information: The security mechanism invoked when you use privileged Server Manager commands depends on the operating system you are using. Most operating systems have a secure authentication mechanism when logging onto the operating system. On these systems, your default operating system privileges will usually determine whether you can use CONNECT INTERNAL, STARTUP, and SHUTDOWN. For operating systems with non-secure authentication mechanisms, you are usually required to enter a password for CONNECT INTERNAL. For more information, see your Oracle system-specific documentation.

The SET INSTANCE and SHOW INSTANCE Commands

You can change the default instance with the Server Manager command SET INSTANCE instance_path, where instance_path is a valid SQL*Net connect string (without a user ID/password). If you are connected to an instance, you must disconnect before using SET INSTANCE. Alternatively, if you do not wish to disconnect from the current instance, you may use the CONNECT command with instance_path.

You can use the SET INSTANCE command to specify an instance on a remote node for the commands STARTUP and SHUTDOWN. The parameter file for a remote instance must be on the local node.

The SHOW INSTANCE command displays the connect string for the default instance. SHOW INSTANCE returns the value local if you have not used SET INSTANCE during the Server Manager session.

To reset to the default instance, use SET INSTANCE without specifying a connect string or specify LOCAL (but not DEFAULT, which would indicate a connect string for an instance named "DEFAULT").

The following Server Manager line mode examples illustrate the relationship between SHOW INSTANCE and SET INSTANCE:

SHOW INSTANCE 
Instance                      local 

SET INSTANCE node1 
Oracle7 Server Release 7.3.2 - Production
With the distributed, parallel query and Parallel Server options
PL/SQL V2.2.2.0.0 - Production

SHOW INSTANCE 
Instance                      node2 

SET INSTANCE 
ORACLE7 Server Release 7.3.2 - Production
With the procedural, distributed, and Parallel Server options 
PL/SQL V2.0.18.1.0 - Production
SHOW INSTANCE 
Instance                      local 
SET INSTANCE DEFAULT 
ORA-06030: NETDNT: connect failed, unrecognized node name 

The CONNECT Command

The CONNECT command can associate Server Manager with either the default instance or an instance which you specify explicitly. The instance to which Server Manager connects becomes the current instance.

The CONNECT command has the following syntax:

where instance-path is a valid SQL*Net connect string. CONNECT without the argument @instance-path connects to the default instance (which may have been set previously with SET INSTANCE).

Connecting as SYSOPER or SYSDBA allows you to perform privileged operations, such as instance startup and shutdown.

Multiple Server Manager sessions can connect to the same instance at the same time. When you are connected to one instance, you can connect to a different instance without using the DISCONNECT command. Server Manager disconnects you from the first instance automatically whenever you connect to another one.

The CONNECT @instance-path command allows you to specify an instance before using the Server Manager commands MONITOR, STARTUP, SHUTDOWN, SHOW SGA, and SHOW PARAMETERS.

See Also: Oracle Server Manager User's Guide for syntax of Server Manager commands.

Oracle Network Manager Administrator's Guide for the proper specification of instance_path.

Oracle7 Server Administrator's Guide for information on connecting with SYSDBA or SYSOPER privileges.

Using a Password File to Authenticate Users on Multiple Instances

You can use a password file to authenticate users performing database administration when running multiple instances on a parallel server. In this case, the environment variable for each instance must point to the same password file. Similarly, the REMOTE_LOGIN_PASSWORDFILE initialization parameter for each instance must be set to the appropriate, identical value.

See Also: Oracle7 Server Administrator's Guide for information about the REMOTE_LOGIN_PASSWORDFILE parameter.

For more information on the exact name of the password file, or for the name of the environment variable used to specify this name for your operating system, see your Oracle system-specific documentation.


Shutting Down Instances

Use the following procedure to shut down an instance:

To Shut Down an Instance Using SQL

	CONNECT username/password AS SYSDBA

	ALTER DATABASE database_name CLOSE

	ALTER DATABASE database_name DISMOUNT

Alternatively, you can use the Server Manager command SHUTDOWN, which performs all three of these steps for the current instance.

In a parallel server, shutting down one instance does not interfere with the operations of any instances still running.

To shut down a database which is mounted in shared mode, you must shut down every instance in the parallel server. The parallel server allows you to shut down instances in parallel from different nodes. When an instance shuts down abnormally, Oracle forces all user processes running in that instance to log off the database. If a user process is currently accessing the database, Oracle terminates that access and returns the message "ORA-1092: Oracle instance terminated. Disconnection forced". If a user process is not currently accessing the database when the instance shuts down, Oracle returns the message "ORA-1012: Not logged on" upon the next call or request made to Oracle.

After a NORMAL or IMMEDIATE shutdown, instance recovery is not required. Recovery is required, however, after the SHUTDOWN ABORT command or after an instance terminates abnormally. The SMON process of an instance which is still running performs instance recovery for the instance which shut down. If no other instances are running, the next instance to open the database performs instance recovery for any instances which need it.

If multiple Server Manager sessions are connected to the same instance simultaneously, all but one must disconnect before the instance can be shut down normally. You can use the IMMEDIATE or ABORT option of the SHUTDOWN command to shut down an instance when multiple Server Manager sessions (or any other sessions) are connected to it.

See Also: "Starting Up and Shutting Down" in Oracle7 Server Administrator's Guide, for options of the SHUTDOWN command.


Limiting Instances for the Parallel Query Option

Although the parallel query option does not require the Oracle Parallel Server, some aspects of parallel query apply only to a parallel server.

The INSTANCES keyword of the PARALLEL clause of the CREATE TABLE, ALTER TABLE, CREATE CLUSTER, and ALTER CLUSTER commands allows you to specify that a table or cluster is split up among the buffer caches of all available instances of a parallel server when the table is scanned in a parallel query.

If you do not want tables to be dynamically partitioned among all the available instances, you can specify the number of instances that can participate in scanning or caching with the parameter PARALLEL_DEFAULT_MAX_INSTANCES or the ALTER SYSTEM command.

If you want to specify the number of instances to participate in parallel query processing at startup time, you can specify a value for the initialization parameter PARALLEL_DEFAULT_MAX_INSTANCES.

If you want to limit the number of instances available for parallel query processing dynamically, use the ALTER SYSTEM command. For example, if your parallel server has ten instances running, but you want only eight to be involved in parallel query processing, while the remaining two instances will be dedicated for other use, you can issue the following command:

ALTER SYSTEM SET SCAN_INSTANCES = 8;

Thereafter, if a table's definition has a value of ten specified for the INSTANCES keyword, the table will be scanned by query servers on only eight of the ten instances. Oracle selects the first eight instances in this example. You can set the initialization parameter PARALLEL_MAX_SERVERS to zero on the instances that you do not want to participate in parallel query processing.

If you wish to limit the number of instances that cache a table, you can issue the following command:

ALTER SYSTEM SET CACHE_INSTANCES = 8;

Thereafter, if a table definition has 10 specified for the INSTANCES keyword and the CACHE keyword was specified, the table will be divided evenly among eight of the ten available instances' buffer caches.

See Also: "Initialization Parameters" [*].

Oracle7 Server Reference for more information about parameters.




Go to previous file in sequence Go to next file in sequence
Prev Next
Oracle
Copyright © 1996 Oracle Corporation.
All Rights Reserved.
Go to Product Documentation Library
Library
Go to books for this product
Product
Go to Contents for this book
Contents
Go to Index
Index