Software Updater in Kaspersky Total Security is one of the best feature which is really helpful. It's automated updates of the software applications saves time and makes things easier.

However, I did come across an issue with Oracle VM VirtualBox. When this tool is automatically updated, it restarts the computer without approval. This mean all your opened windows get closed. Furthermore, sometimes when Oracle VM VirtualBox is automatically updated, the virtual environments created in this tool does not open and shows an error. To fix this issue, you need to do a clean manual reinstallation of Oracle VM VirtualBox and add the virtual environments again.

This is in Windows 10.

Moreover, the last time Oracle VM VirtualBox was automatically updated by Kaspersky Total Security, I did not have the above-mentioned issue. But, it still restarts the computer without confirmation.

My assumption is that when Oracle VM VirtualBox gets automatically update, the Oracle VM VirtualBox Extension Pack does not get updated. This could be a possible issue with virtual environments not opening.

Oracle Virtualization Community, https://community.oracle.com/community/groundbreakers/server_%26_storage_systems/virtualization

• For additional information on best practices regarding Oracle VM deployments, contact Oracle Support and refer to Document ID 1940756.1.

13.1 Capturing Diagnostic Information for Oracle VM

Oracle Support Services provide a script,

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

7, to collect diagnostic information from your Oracle VM environment for troubleshooting purposes. This script is installed with Oracle VM Manager in the following location:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

8

Syntax

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

9 {

service iscsi restart

0 } [

service iscsi restart

1 ]

Options

Option

Description

{

service iscsi restart

0 }

Specifies the user that runs the script. You should not specify a user other than the default Oracle VM Manager user,

service iscsi restart

3.

[

service iscsi restart

4 ]

Lists all servers for which the script can collect diagnostic information. If you specify this option, the script displays the list of servers that the Oracle VM Manager owns and then exits. You must then run the script again to collect diagnostic information.

[

service iscsi restart

5 ]

Specifies at least one instance of Oracle VM Server for which you want to collect diagnostic information. Use this option to exclude servers in your environment or limit the diagnostic collection to certain servers only.

Separate multiple server names with a comma.

Tip

It can take the script several minutes to collect diagnostic information across your Oracle VM environment. To save time, specify this option to include only the instances of Oracle VM Server for which issues might exist, especially if you have a large number of Oracle VM Server deployments.

Examples

To collect diagnostic information for your Oracle VM environment, including Oracle VM Manager and all of the owned Oracle VM Servers in the model, run:

./vmpinfo3.sh --username=admin

To list the servers for which you can collect diagnostic information, run:

./vmpinfo3.sh --username=admin listservers

To collect diagnostic information for specific servers only, run:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

Output

The script creates a tarball in the

service iscsi restart

6 directory that contains log files, sosreports, and other diagnostic information such as details about the Oracle VM model.

When the script completes successfully, it displays the filename of the tarball, as in the following example:

---

Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

---

13.2 Troubleshooting Oracle VM Server

This section describes some problems you may encounter when using , and explains how to resolve them.

If you need to contact Oracle Support Services, you will be asked to supply the log files mentioned in this section. You may also be required to provide the exact version of each Oracle VM component. To find the version of , click the Help menu, thenAbout. To find the version of Oracle VM Server and , see the Control Domain Perspective section of the Oracle VM Manager User's Guide .

13.2.1 Oracle VM Server Debugging Tools

If creation fails, check the Oracle VM Server log files and use the command-line tools to help you find the cause of a problem. There are a number of useful command-line tools, important directories, and log files that you should check when troubleshooting problems with Oracle VM Server. This section discusses these tools and log files.

13.2.1.1 Oracle VM Server Directories

The important Oracle VM Server directories you should check when troubleshooting problems with Oracle VM Server are listed in .

Table 13.1 Oracle VM Server directories

Directory

Purpose

service iscsi restart

7

Contains Oracle VM Server configuration files for the Oracle VM Server daemon and virtualized .

service iscsi restart

8

Contains networking related scripts.

service iscsi restart

9

Contains the following files: .

• iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

  iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

  0, log file for the Oracle VM Agent.

• iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

  iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

  1, logs virtual machine life cycle .

• iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

  iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

  2, logs remote VNC console access, and all communication with Oracle VM Manager.

• iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

  iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

  3, logs all Oracle VM Server messages.

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

4

Contains Oracle VM Server log files.

13.2.1.2 Oracle VM Server Log Files

The Oracle VM Serverlog files you should check when troubleshooting problems with Oracle VM Server are listed in the following table:

Table 13.2 Oracle VM Server log files

Log File

Directory

Description

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

5

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

6

Contains a log of all the actions of the Oracle VM Server daemon. Actions are normal or error conditions. This log contains the same information as output using thexm log command.

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

7

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

6

Contains more detailed logs of the actions of the Oracle VM Server daemon.

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

9

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

6

Contains a log of hotplug events. Hotplug events are logged if a device or network script does not start up or become available.

service iscsid restart

1.log

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

6

Contains a log for each hardware virtualized guest. This log is created by the quemu-dm process. Use the ps command to find the pid (process identifier) and replace this in the file name.

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

0

service iscsid restart

4

Contains a log for Oracle VM Agent.

service iscsid restart

5

service iscsid restart

4

Contains a log for Oracle VM Storage Connect plug-ins.

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

2

service iscsid restart

4

Contains a log for the Oracle VM virtual machine console.

iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

1

service iscsid restart

4

Contains a log for the Oracle VM watch daemon.

13.2.1.3 Oracle VM Server Command Line Tools

The following table lists command line tools you can use when troubleshooting problems with Oracle VM Server:

Note

These command line tools are included as part of the Xen environment. You should refer to the appropriate Xen documentation for more information on using them.

Table 13.3 Oracle VM Server command line tools

Command Line Tool

Description

xentop

Displays real-time information about Oracle VM Server and domains.

xm dmesg

Displays log information on the .

xm log

Displays log information of the Oracle VM Server daemon.

13.2.2 Using DHCP on Oracle VM Servers

It is recommended that you install Oracle VM Server on a computer with a static IP address. If you use DHCP to manage the IP address space in your environment, the DHCP server should be configured to map the server interface MAC addresses to specific IP assignments. This makes sure your host always receives the same IP address. The behavior of the Oracle VM Server host is undefined if used in an environment where your IP address may change due to DHCP lease expiry.

13.2.3 Cannot Use Certain Key Combinations When Connecting to Dom0 Console

Some server models and some client terminals are not ideally compatible with regard to special key combinations. For instance, on some HP servers, such as the HP DL380G4 (BIOS P51) server, the Alt-F2 key combination required to toggle to the login screen does not work for all terminal clients. Some terminal clients provide alternate key mappings, so it is worth checking the documentation of your selected terminal client to determine whether an alternative mapping is available.

If you are using the Windows PuTTY SSH client, you can pressAlt + the right arrow key andAlt + the left arrow key to toggle the login screen, instead of the printed Alt-F2.

13.2.4 Storage Array LUN Remapping on Oracle VM Servers

Storage array LUN remapping is not supported by Oracle VM Servers. An Oracle VM Server must maintain the connections to a storage array's logical drive using the same LUN IDs. If a LUN is remapped, the following error may be printed in the Oracle VM Server's messages log:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

To work around this issue:

• For Fibre Channel storage, reboot the Oracle VM Server. The new storage array LUN IDs are used.
• For iSCSI storage, restart the iscsi daemon on the Oracle VM Server to delete and restore all iSCSI target connections, for example:

  service iscsi restart

  Alternatively, on the Oracle VM Server, log out and log in again to the target for which the LUN IDs have changed, for example:

  iscsiadm --mode node --logout ipaddress iqn.xyz:1535.uuid

  iscsiadm --mode node --login ipaddress iqn.xyz:1535.uuid

13.2.5 Tuning ISCSI Settings on Oracle VM Servers

In some cases, it is possible to run into limitations or bugs within a particular ISCSI implementation that may require you to tune your ISCSI settings for storage initiators on each of your Oracle VM Servers.

Typically this is required when you experience an IO lock on a LUN and an unexpected change in the kernel workload on the Oracle VM Server. You may also notice a dramatic increase in network traffic between the Oracle VM Server and the storage array. This particular case has been noted to occur on some ZFS appliances running Oracle Solaris 11 and is related to the Sun iSCSI COMSTAR port provider. The problem can usually be resolved by updating package versions, however if this is not an option you may tune your ISCSI settings on each Oracle VM Server that communicates with a SUN.COMSTAR target to enable flow control.

To tune ISCSI, on each Oracle VM Server, perform the following steps:

• Open timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

   (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB  
  

  1 in a text editor.
• Locate the section titled timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

   (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB  
  

  2.
• Change the value of the entry timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

   (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB  
  

  3 to timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

   (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB  
  

  4, and the value of the entry timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

   (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB  
  

  5 to timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

   (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB  
  

  6.
• Save the file.
• Restart the ISCSI daemon by issuing the following command:

  service iscsid restart

Note

The preferred resolution to this issue is to update your software. Manual configuration of Oracle VM Server settings is not generally advisable, as changes may be lost during future updates.

13.2.6 Troubleshooting Clustered Server Pools Oracle VM Server for x86

There are some situations where removing an Oracle VM Server from a server pool may generate an error. Typical examples include the situation where an OCFS2-based repository is still presented to the Oracle VM Server at the time that you attempt to remove it from the server pool, or if the Oracle VM Server has lost access to the server pool file system or the heartbeat function is failing for that Oracle VM Server. The following list describes steps that can be taken to handle these situations.

• Make sure that there are no repositories presented to the server when you attempt to remove it from the server pool. If this is the cause of the problem, the error that is displayed usually indicates that there are still OCFS2 file systems present. See the Repositories Perspective section in the Oracle VM Manager Online Help for more information.
• If a pool file system is causing the remove operation to fail, other processes might be working on the pool file system during the unmount. Try removing the Oracle VM Server at a later time.
• In a case where you try to remove a server from a clustered server pool on a newly installed instance of Oracle VM Manager, it is possible that the file server has not been refreshed since the server pool was discovered in your environment. Try refreshing all storage and all file systems on your storage before attempting to remove the Oracle VM Server.
• In the situation where the Oracle VM Server cannot be removed from the server pool because the server has lost network connectivity with the rest of the server pool, or the storage where the server pool file system is located, a critical event is usually generated for the server in question. Try acknowledging any critical events that have been generated for the Oracle VM Server in question. See the Events Perspective section in the Oracle VM Manager Online Help for more information. Once these events have been acknowledged you can try to remove the server from the server pool again. In most cases, the removal of the server from the server pool succeeds after critical events have been acknowledged, although some warnings may be generated during the removal process. Once the server has been removed from the server pool, you should resolve any networking or storage access issues that the server may be experiencing.
• If the server is still experiencing trouble accessing storage and all critical events have been acknowledged and you are still unable to remove it from the server pool, try to reboot the server to allow it to rejoin the cluster properly before attempting to remove it again.
• If the server pool file system has become corrupt for some reason, or a server still contains remnants of an old stale cluster, it may be necessary to completely erase the server pool and reconstruct it from scratch. This usually involves performing a series of manual steps on each Oracle VM Server in the cluster and should be attempted with the assistance of Oracle Support.

13.2.7 Allocating Memory for Multiple Infiniband HCAs

Out of memory errors can occur when using multiple Infiniband host channel adapters (HCA) with the PCIe Scalable Interface (psif) driver for paravirtualized environments on Oracle VM Server. These out of memory errors occur because the psif driver requires a minimum 30 MB of memory for the driver itself in addition to 50 MB of memory for each interface instance. The default memory allocation for dom0 does not provide enough memory to support multiple interfaces.

To resolve the out of memory errors, you should increase the dom0 memory allocation for Oracle VM Server. See .

The following is an example of an out of memory error that is written to

timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

 (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB

7:

timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

 (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB

13.2.8 Resolving Issue Where NIC Fails to Get IP Address if Configured for DHCP

In some cases, when you configure a network interface for Oracle VM Server to retrieve IP addresses via DHCP, and then bring up that interface, it cannot retrieve an IP address and the following error occurs:

Determining IP information for interfacename... failed; no link present. Check cable?

This issue can be caused when the time it takes the network interface to come up is greater than the time set for the

timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

 (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB

8 environment variable.

To resolve this issue, set the value of the

timestamp hostname kernel: [ 1465.466059] Out of memory: Kill process 8106 (python) score 0 or sacrifice child Connection to hostname closed.l: [ 1465.466089] Killed process 8467

 (ovs-agent) total-vm:108488kB, anon-rss:12kB, file-rss:2064kB

8 to a value of 10 or higher in the

Determining IP information for interfacename... failed; no link present. Check cable?

0 file, as in the following example:

./vmpinfo3.sh --username=admin listservers

0

13.3 Troubleshooting Oracle VM Manager

This section describes some problems you may encounter when using , and explains how to resolve them.

13.3.1 Oracle VM Manager Log Files

Oracle VM Manager error messages are displayed in the user interface, in the Jobs tab, in the object's list and are also available in log files. Log files are stored in the following directory on the Oracle VM Manager host computer:

Determining IP information for interfacename... failed; no link present. Check cable?

1

Oracle VM Manager that you can use for troubleshooting are as follows:

Log file

Description

Determining IP information for interfacename... failed; no link present. Check cable?

2

Tracks HTTP access to the Web interface of the Oracle VM Manager and to the underlying HTTP interface. This log can be used to track access and HTTP operations within Oracle VM Manager to help debug access issues and to audit access to the Oracle VM Manager.

Determining IP information for interfacename... failed; no link present. Check cable?

3

Tracks events within the underlying Oracle WebLogic Server framework, including events triggered by Oracle VM Manager. This log can be used to track a variety of issues within Oracle VM Manager including TLS/SSL certificate issues, server availability issues, and any actions performed within Oracle VM Manager which are usually identifiable by searching for items containing the string

Determining IP information for interfacename... failed; no link present. Check cable?

4. Log in failures resulting from locked accounts (as opposed to incorrect credentials) are also in this file.

Determining IP information for interfacename... failed; no link present. Check cable?

5

Tracks exceptions within the underlying Oracle WebLogic Server framework, including particular events triggered by Oracle VM Manager such as log in failures due to incorrect credentials. This log can be used to track Oracle VM Manager behavior that results in an exception or for log in failure, which can be tracked by searching for the string

Determining IP information for interfacename... failed; no link present. Check cable?

6.

Because log file format is determined by Oracle WebLogic Server, many of these files may be difficult to read. A log parsing tool is included with Oracle VM Manager to help extract useful information from the actual log files. The log parsing tool is named

Determining IP information for interfacename... failed; no link present. Check cable?

7 and is located at:

Determining IP information for interfacename... failed; no link present. Check cable?

8

Determining IP information for interfacename... failed; no link present. Check cable?

7 can do the following things:

• Convert and combine all the AdminServer log files into one easier-to-read file.
• Create a filtered summary log file that only lists errors.
• Tail the AdminServer log, applying the filtering on the fly.

Usually analysis of the logs starts by generating an error summary log. The summary file can act as an index into the filtered file to investigate and analyze errors, providing you with time stamps an a shortened summary of each error that may need further investigation. To generate a summary log file, do the following:

./vmpinfo3.sh --username=admin listservers

1

This generates a file named

./vmpinfo3.sh --username=admin listservers

00 in the local directory. You can use this to look forerrors that occurred within Oracle VM Manager.

To get a full log of all events and errors within Oracle VM Manager you can do the following:

./vmpinfo3.sh --username=admin listservers

2

This generates a file named

./vmpinfo3.sh --username=admin listservers

01 in the local directory. You can use this to look for all events that occurred within Oracle VM Manager.

Finally, you can use

Determining IP information for interfacename... failed; no link present. Check cable?

7 to filter results on the fly while tailing the log:

./vmpinfo3.sh --username=admin listservers

3

UseCtrl+C to exit the program when you have finished tailing the log file.

13.3.2 Oracle VM Manager Web Interface Database Synchronization

The Oracle VM Manager Web Interface and the Oracle VM Manager Command Line Interface both use a representation of the Oracle VM data model to to more quickly retrieve data objects, which optimizes the performance of Oracle VM Manager. The representation of the data model is saved to a separate database to the primary Oracle VM Manager database. This separate database relies on events from Oracle VM Manager to stay synchronized with the actual data model.

It is possible, in specific cases, for the representation of the data model to become out of sync with the actual data model. As a result, some objects in the Oracle VM Manager Web Interface do not reflect the actual environment. A typical scenario where this may happen is during a virtual machine clone operation that fails. During this process, the virtual machine is actually created within the data model and the database used by the user interface layer. If a part of the whole operation fails, Oracle VM Manager attempts to clean up and roll back, resulting in the virtual machine being removed from the data model. However, in this case, an event is not generated to notify that the clean up has succeeded, and the virtual machine information remains in the user interface database. The result is that the cloned virtual machine is still shown in the Oracle VM Manager Web Interface and the Oracle VM Manager Command Line Interface even though it does not actually exist in the environment.

The user interface database is not resynchronized automatically when the service is restarted, as this can take a long time for large environments. To force database resynchronization, you must create a file on the Oracle VM Manager host before restarting the service. The following instructions explain how to force resynchronization.

Steps to resynchronize the Oracle VM Manager Web Interface database

• You must access the shell of the Oracle VM Manager host, either directly or over SSH.
• Change user to the 'oracle' user, using su.

  ./vmpinfo3.sh --username=admin listservers

  4
• Touch a file called

  ./vmpinfo3.sh --username=admin listservers

  03 as the oracle user.

  ./vmpinfo3.sh --username=admin listservers

  5 If you are unable to do this as the oracle user, then touch the file as any other user but ensure that its permissions are such that any other user can delete the file:

  ./vmpinfo3.sh --username=admin listservers

  6
• Restart the Oracle VM Manager service as root:

  ./vmpinfo3.sh --username=admin listservers

  7

13.3.3 Increasing the Memory Allocated to Oracle WebLogic Server

In environments that host thousands of virtual machines and where large data sets exist, performance issues can occur with Oracle VM Manager, such as garbage collection taking a long time or out of memory conditions arise. Likewise, Oracle VM Manager can generally be slow to respond and a significant performance degradation occurs. To resolve these performance issues, you can increase the amount of memory that is allocated to Oracle WebLogic Server.

Note

You should not increase the memory allocation to the maximum limit that is available to the host server. As a guideline, you should leave at least 2GB available memory for the host operating system and other services.

To increase the memory allocated to Oracle WebLogic Server, do the following:

1. Start an ssh session to the Oracle VM Manager as the root user.
2. Open the following file for editing:

   ./vmpinfo3.sh --username=admin listservers

   04
3. Specify the amount of memory allocated to Oracle WebLogic Server as the value for the

   ./vmpinfo3.sh --username=admin listservers

   05 property. Note The default is

   ./vmpinfo3.sh --username=admin listservers

   06.
4. Save and close

   ./vmpinfo3.sh --username=admin listservers

   04.
5. Restart the Oracle VM Manager service.

   ./vmpinfo3.sh --username=admin listservers

   7

13.3.4 No File Systems Found When Searching a Storage Server

On storage servers that have a very large number of file systems available, the UI may time out while refreshing the list of available file systems, resulting in a No File Systems Available message. This usually means that the time out value is set too low for the number of file systems that the UI needs to refresh. To resolve this, change the settings for theRefresh Timeout Value in thePreferences Pane on theTools and Resources tab in the Oracle VM Manager Web Interface to increase the timeout value.

See the Preferences section in the Oracle VM Manager Online Help for more information on Oracle VM Manager UI preferences.

13.3.5 Cannot Discover Servers to Oracle VM Manager Due To Time Differences

Oracle VM Manager uses certificate-based authentication to maintain secure communication with the Oracle VM Agent that runs on each Oracle VM Server instance. This means that the system time on each Oracle VM Server must be within the bounds of the certificate valid from and valid to timestamps, or certificate validity is challenged and Oracle VM Manager is unable to authenticate to the Oracle VM Agent.

In most instances, this is not a problem, since servers are automatically configured to use the Oracle VM Manager as an NTP server as soon as ownership is taken. However, during server discovery the Oracle VM Manager uses a password to perform its initial authentication against a server and to provide its certificate for ongoing communication. During this phase of discovery, a check is performed to ensure that the system time on the Oracle VM Server is within the bounds required for certificate authentication to take place. If this is not the case, discovery fails and an error message is returned:

./vmpinfo3.sh --username=admin listservers

9

In this situation, it is necessary that you access the affected Oracle VM Server directly and update its system time manually before attempting to rediscover the server using Oracle VM Manager. Once Oracle VM Manager has completed discovery and is able to take ownership of the server, its NTP configuration is updated automatically and it remains synchronized with the Oracle VM Manager host.

13.3.6 Cannot Create a Clustered Server Pool on a Disk that already has an OCFS2 File System

If you attempt to create a clustered server pool using a disk located on a storage device that already contains an OCFS2 file system, the Oracle VM Agent on the server detects the file system and refuses to overwrite it. This is normal behavior and protects you from accidentally setting up two OCFS2 file systems with matching UUIDs on the same disk, leading to instability and unexpected behavior.

If you are certain that the existing OCFS2 file system that is already present on the disk is no longer in use by any other server pool or repository, you can clean the disk by connecting to the Oracle VM Server and issuing the following command:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

0

Replace

./vmpinfo3.sh --username=admin listservers

08 with the correct path to the disk device where you are creating the new server pool cluster.

Warning

Using dd is data destructive. Make sure you are certain about the disk device name and that the OCFS2 file system that you are deleting is no longer in use. It is advisable to make backups of any data that exists on the disk that you are editing before proceeding. This operation should be performed by a skilled systems administrator.

13.3.7 Cannot Create a Repository on a Device that has Partitions

A repository cannot be hosted on a physical disk that has pre-existing partitions. If you attempt to create a repository on a disk that already has a partition, an error is generated notifying you that the backing device is not allowed to contain partitions. If you are intent on creating a repository on the selected disk, you must delete any pre-existing partitions. This may require you to directly access the Oracle VM Server where the disk is mounted and to manually remove the partition objects on the disk using the fdisk command. For example:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

1

Warning

Using fdisk is data destructive. Make sure you are certain about the disk device name and partitions that you are deleting. It is advisable to make backups of any data that exists on the disk that you are editing before proceeding. This operation should be performed by a skilled systems administrator.

Usually, restarting the affected Oracle VM Server after performing these operations is advisable. In the case of an iSCSI disk, connections to targets need to be re-initiated. In all cases, the storage needs to be refreshed. Simply restarting the Oracle VM Server ensures that all necessary actions are performed.

Once the Oracle VM Server has restarted, you may attempt to recreate the repository.

13.3.8 Removing Oracle VM Template Configuration Packages on Oracle Linux 5 Hosts

If you are running Oracle VM Manager on an Oracle Linux 5 host, removing all

./vmpinfo3.sh --username=admin listservers

09 related RPM packages in a single command results in an error and some packages are not successfully removed.

To resolve this, you must use two yum erase operations and ensure that

./vmpinfo3.sh --username=admin listservers

09 is removed last, as follows:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

2

13.3.9 Unable to Manage Oracle VM Server for x86 at Release 3.2.10 or 3.2.11, and Oracle VM Agent for SPARC at Release 3.3.1

As of Oracle VM Release 3.4.5, Oracle VM Manager uses the TLS version 1.2 (TLSv1.2) protocol for all connections for security reasons. As a result, management of Oracle VM Server for x86 at Release 3.2.10 or 3.2.11, and Oracle VM Agent for SPARC at Release 3.3.1, is not possible by default. As a workaround, you must enable the TLSv1 protocol, which is less secure.

For instructions on how to do this, see Enabling the TLS Version 1 Protocol in the Oracle VM Installation and Upgrade Guide .

13.4 Troubleshooting Virtual Machines

The section contains information on known issues you may encounter when creating or using , and explains how to resolve them.

13.4.1 Setting the Guest's Clock

guests may perform their own system clock management, for example, using the NTPD (Network Time Protocol daemon), or the may perform system clock management for all guests.

You can set guests to manage their own system clocks by setting the

./vmpinfo3.sh --username=admin listservers

11 parameter to

./vmpinfo3.sh --username=admin listservers

12 in the

./vmpinfo3.sh --username=admin listservers

13 file. For example:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

3

If you want to set the hypervisor to manage paravirtualized guest system clocks, set

./vmpinfo3.sh --username=admin listservers

11 to

./vmpinfo3.sh --username=admin listservers

15. Any attempts to set or modify the time in a guest will fail.

You can temporarily override the setting in the

./vmpinfo3.sh --username=admin listservers

16 file. For example:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

4

Note

This setting does not apply to hardware virtualized guests.

13.4.2 Wallclock Time Skew Problems

Additional parameters may be needed in the boot loader (

./vmpinfo3.sh --username=admin listservers

17. configuration file for certain operating system variants after the guest is installed. Specifically, for optimal clock accuracy, Linux guest boot parameters should be specified to ensure that thepit clock source is utilized. Adding

./vmpinfo3.sh --username=admin listservers

18 for most guest will result in the selection of pit as the clock source for the guest. Published templates for Oracle VM include these additional parameters.

Proper maintenance of virtual time can be tricky. The various parameters provide tuning for virtual time management and supplement, but do not replace, the need for anntp time service running within guest. Ensure that the

./vmpinfo3.sh --username=admin listservers

19 service is running and that the

./vmpinfo3.sh --username=admin listservers

20 configuration file is pointing to valid time servers.

13.4.3 Mouse Pointer Tracking Problems

If your mouse pointer fails to track your cursor in a VNC Viewer session in a hardware virtualized guest, add the following to the Oracle VM Server configuration file located at

./vmpinfo3.sh --username=admin listservers

21 to force the device model to use absolute (tablet) coordinates:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

5

Restart the Oracle VM Server for the changes to take effect. You may need to do this for each Oracle VM Server in the .

13.4.4 Cloning Virtual Machine from Oracle VM 2.

./vmpinfo3.sh --username=admin listservers

22 Template Stuck in Pending

When creating a virtual machine from an Oracle VM 2.

./vmpinfo3.sh --username=admin listservers

22 template, the job fails with the error:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

6

This is caused by a network configuration inconsistency with the

./vmpinfo3.sh --username=admin listservers

24 entry in the virtual machine's configuration file.

To resolve this issue, remove any existing networks in the , and replace them with valid networks which have the Virtual Machine role. Start the clone job again and the virtual machine clone is created. Alternatively, remove any existing networks in the template, restart the clone job, and add in any networks after the clone job is complete.

13.4.5 Hardware Virtualized Guest Stops

When running hardware virtualized guests, the process (qemu-dm) may have its memory usage grow substantially, especially under heavy I/O loads. This may cause the hardware virtualized guest to stop as it runs out of memory. If the guest is stopped, increase the memory allocation for , for example from 512 MB to 768 MB. See for information on changing the dom0 memory allocation.

13.4.6 Migrating Virtual Machines

You cannot on computers with hardware that is not identical. To migrate virtual machines, you must have hardware that is the same make and model and the CPU must be in the same CPU family.

Virtual machines can be live migrated between instances of Oracle VM Server that are at the same release or later. For virtual machines running on an x86 platform, a rule exception is generated if you attempt to live migrate a virtual machine to an Oracle VM Server with an earlier release than the Oracle VM Server where the virtual machine is running.

13.4.7 Recovering From A Failed Local Virtual Machine Migration

In the event where a virtual machine hosted on a local repository is live migrated and the migration source, or target, Oracle VM Server becomes unavailable during the migration, Oracle VM Manager attempts to perform a rollback of the operation. This rollback process brings the original version of the virtual machine back online on the source Oracle VM Server and then performs a cleanup operation on the target Oracle VM Server when it becomes available again. This cleanup process involves killing the paused virtual machine that may have been copied to the target Oracle VM Server and then cleaning the target repository of virtual disks, virtual machine configurations and temporary files. Finally a repository refresh is performed on the repository on the source server to ensure that everything is in order.

Before the cleanup operation is triggered, an event is created within Oracle VM Manager to indicate that the migration job has failed or been aborted and to track the rollback process. When the event is generated within Oracle VM Manager, it is set with a 'WARNING' status. The rollback process is generated as a set of up to three different jobs that are each given a timeout period of 15 minutes, and which are triggered to attempt to run every 10 seconds. If these jobs succeed, Oracle VM Manager acknowledges the event. If the jobs all timeout, Oracle VM Manager still acknowledges the event, but a second user-acknowledgeable event is created with 'WARNING' status to indicate that the rollback failed. Depending on the cause of the rollback failure, Oracle VM Manager might also create user-acknowledgeable events with 'CRITICAL' status.

Because jobs are usually performed sequentially, it may take a total of 45 minutes before the entire rollback process times out and the new event indicating rollback failure is generated. The rollback failure event is also logged in the the log file

./vmpinfo3.sh --username=admin listservers

25 on the Oracle VM Manager host.

The information in the rollback failure event contains the rollback plan that Oracle VM Manager attempted to follow to cleanup a failed virtual machine migration. This event can be viewed using the getEventsForObject command with the Oracle VM Manager Command Line Interface, by viewing the events associated with the virtual machine within the Oracle VM Manager Web Interface or via the Oracle VM Web Services API.

The following content represents the typical output displayed within the description field for a rollback failure event:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

7

Note that the description of the event provides detailed information about the migration process and indicates that the migration job has failed. The message explains that the virtual machine is set back to run on the source server and that the source virtual disks have been retained. This means that the virtual machine may either be running or stopped on the source server, but from the perspective of Oracle VM Manager, the location of the virtual machine has been reverted. Most significantly, the output contains a 'post-migration completion plan'. This plan provides a full breakdown of the steps that must be performed to roll the environment back to its original state.

If an event like this appears for a failed migration of a locally hosted virtual machine, you must manually perform the rollback steps on the target server when it next becomes available. It is very important that you ensure that the rollback steps are performed on the systems indicated in the post-migration completion plan. Performing any of these steps on another server could have detrimental effects and could result in virtual machine corruption.

Kill the Virtual Machine on the Indicated Oracle VM Server

The first step in this plan involves killing the virtual machine on the indicated Oracle VM Server or servers. Depending on the state of the migration at the time that the target Oracle VM Server became unavailable, this may be require an action on either the target Oracle VM Server or both the target and source Oracle VM Servers. In some cases you may not need to perform this action on either Oracle VM Server. The appropriate action is logged in the event description.

During the migration, the virtual machine enters into a paused state as it is copied from the source Oracle VM Server to the target Oracle VM Server. Once the copy is complete, the virtual machine on the target Oracle VM Server is not indicated within Oracle VM Manager in any way, as this would conflict with the virtual machine with the identical UUID that is located on the original source Oracle VM Server. This transition is performed within Oracle VM Manager when the migration is complete. As a result two virtual machines with identical UUIDs may exist within the environment for the period of the migration. If the target server goes offline at any point during the migration, it is frequently the case that at least one of these virtual machines must be killed off to prevent conflict. Since the representation of the virtual machine within Oracle VM Manager is not reliable until the rollback has been completed, it is necessary that you must perform the kill operation directly on the indicated Oracle VM Server. This is usually done over SSH as the root user, using the following command:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

8

Note that

./vmpinfo3.sh --username=admin listservers

26 should match the UUID of the virtual machine that you were originally migrating. Also pay attention to the quotes in each of the arguments presented here. The first argument for this command is empty, so a pair of single quotes are enclosed in a pair of double-quotes. The second argument is the UUID of the virtual machine that you intend to kill and is represented as enclosed in a pair of single quotes within a pair of double-quotes. Finally, the last argument is used to force the action and contains the text True enclosed in a pair of double-quotes.

Note that you should use this command to stop the virtual machine because it helps to identify the correct virtual machine domain to destroy, it maintains the integrity of your environment and logs any actions carried out on the virtual machine. Do not attempt to use Xen hypervisor tools to perform any actions on the virtual machine directly without explicit instruction from an Oracle Support representative.

Remove any Virtual Disks from the Repository on the Target Oracle VM Server

A live migration of a virtual machine that is hosted on local storage also requires that any virtual disks are copied from the repository hosted on the source server across to the repository of the target server. Therefore, it is necessary that you manually delete any of these files from the repository hosted on the target Oracle VM Server to clean the environment. To do this, you must SSH to the target Oracle VM Server and delete the files listed in the plan returned in the event description. For example:

./vmpinfo3.sh --username=admin servers=myserver1.example.com,myserver2.example.com

9

Remove the Virtual Machine Configuration from the Repository on the Target Oracle VM Server

The virtual machine configuration for the virtual machine is also copied from the repository hosted on the source server across to the repository of the target server during the migration. Therefore, it is necessary that you manually delete any of these files and directories from the repository hosted on the target Oracle VM Server to clean the environment. To do this, you must SSH to the target Oracle VM Server and delete the files listed in the plan returned in the event description. For example:

---

Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

---

0

Refresh the Repository on the Source Oracle VM Server

During the migration process, Oracle VM Manager updates its model of the source and target repositories hosted on each Oracle VM Server to match the environment as it would be after the migration is complete. It does not revert this representation unless an automated rollback is achieved. If the rollback has failed and you have performed manual steps to revert your environment to its original state, you must also refresh the repository within Oracle VM Manager so that the model accurately reflects the state of the repository. You can either do this using the Oracle VM Manager Web Interface or you can use the Oracle VM Manager Command Line Interface directly. For example:

---

Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

---

1

At this point, your environment should be completely reverted.

13.4.8 Migrating Large Hardware Virtualized Guest Results in CPU Soft Lock

On some hardware, such as the SUN FIRE X4170 M2 Server, migration of very large virtual machines using hardware virtualization can result in a soft lockup causing the virtual machine to become unresponsive. This lock is caused when the migration causes the virtual machine kernel to lose the clock source. Access to the console for the virtual machine shows a series of error messages similar to the following:

---

Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

---

2

To resolve this, the virtual machine must be restarted and the

./vmpinfo3.sh --username=admin listservers

27 option should be added to the HVM guest kernel command line, before rebooting the virtual machine again.

Important

This option should only be used on HVM guest systems that have actually resulted in a CPU soft lock.

13.4.9 Hardware Virtualized Guest Devices Not Working as Expected

Some devices, such as sound cards, may not work as expected in hardware virtualized guests. In a hardware virtualized guest, a device that requires physical memory addresses instead uses virtualized memory addresses, so incorrect memory location values may be set. This is because DMA (Direct Memory Access) is virtualized in hardware virtualized guest.

Hardware virtualized guest operating systems expect to be loaded in memory starting somewhere around address 0 and upwards. This is only possible for the first hardware virtualized guest loaded. Oracle VM Server virtualizes the memory address to be 0 to the size of allocated memory, but the guest operating system is actually loaded at another memory location. The difference is fixed up in the shadow page table, but the operating system is unaware of this.

For example, a sound is loaded into memory in a hardware virtualized guest running Microsoft Windows™ at an address of 100 MB may produce garbage through the sound card, instead of the intended audio. This is because the sound is actually loaded at 100 MB plus 256 MB. The sound card receives the address of 100 MB, but it is actually at 256 MB.

An IOMMU (Input/Output Memory Management Unit) in the computer's memory management unit would remove this problem as it would take care of mapping virtual addresses to physical addresses, and enable hardware virtualized guests direct access to the hardware.

13.4.10 Paravirtualized Guest Disk Devices are Not Recognized

If you opt to create a PVHVM or PVM, you must ensure that all disks that the virtual machine is configured to use are configured as paravirtual devices, or they may not be recognized by the virtual machine. If you discover that a disk or virtual cdrom device is not being recognized by your virtual machine, you may need to edit the

./vmpinfo3.sh --username=admin listservers

28 file for the virtual machine directly. To do this, determine the UUID of the virtual machine, and then locate the configuration file in the repository, for example on an Oracle VM Server:

---

Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

---

3

Locate each

./vmpinfo3.sh --username=admin listservers

29 entry that contains a hardware device such as

./vmpinfo3.sh --username=admin listservers

30,

./vmpinfo3.sh --username=admin listservers

31, or

./vmpinfo3.sh --username=admin listservers

32 and replace with an

./vmpinfo3.sh --username=admin listservers

33 mapping, such as

./vmpinfo3.sh --username=admin listservers

34,

./vmpinfo3.sh --username=admin listservers

35,

./vmpinfo3.sh --username=admin listservers

36 etc.

Restart the virtual machine with the new configuration, to check that it is able to discover the disk or virtual cdrom device.

13.4.11 Cannot Create a Virtual Machine from Installation Media

When creating a virtual machine, the following message may be displayed:

---

Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

---

4

To resolve this issue, make sure the Oracle VM Server supports hardware virtualization. Follow these steps to check:

1. Run the following command to check if hardware virtualization is supported by the CPU:

   ---

   Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

   ---

   5 If any information that contains

   ./vmpinfo3.sh --username=admin listservers

   37 or

   ./vmpinfo3.sh --username=admin listservers

   38 is displayed, it means that the CPU supports hardware virtualization. Here is an example of the returned message:

   ---

   Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

   ---

   6 Note The /proc/cpuinfo command only shows virtualization capabilities starting with Linux 2.6.15 (Intel®) and Linux 2.6.16 (AMD). Use the uname -r command to query your kernel version.
2. Make sure you have enabled hardware virtualization in the BIOS.
3. Run the following command to check if the operating system supports hardware virtualization:

   ---

   Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

   ---

   7 The following is an example of the returned message:

   ---

   Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

   ---

   8

If the CPU does not support hardware virtualization, use the paravirtualized method to create the virtual machine. See the Servers and VMs Tab section in the Oracle VM Manager Online Help for information on creating a .

13.4.12 Cannot Change CD in the Virtual Machine

To change the CD in a virtual machine:

1. Unmount the first CD:

   ---

   Please send /tmp/vmpinfo3-3.x.y.z-timestamp.tar.gz to Oracle OVM support

   ---

   9
2. Select the second ISO file, and clickChange CD.
3. Mount the second CD: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. 0

13.4.13 Generating Guest Dump Files on Oracle VM Server (x86)

The Xen hypervisor makes it possible to generate a core dump file for a virtual machine in the case that it crashes. This file can be useful for debugging and support purposes. Core dump files can be large and to avoid overwriting files, each file is named uniquely. When this facility is enabled, core dump files are saved to

./vmpinfo3.sh --username=admin listservers

39 on the Oracle VM Server where the virtual machine was running when it crashed. This can rapidly use up available disk space on the dom0 system partition. If you enable this facility, you must ensure that enough disk space is available at this path on the Oracle VM Server, either by mounting an additional disk at this path, or by creating a symbolic link for this path to point to an alternate location with plenty of available disk space.

By default, this facility is disable at a system-wide level. It is possible to change this behavior by editing

./vmpinfo3.sh --username=admin listservers

21 directly and changing the lines:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

1

to:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

2

After making this change, you must reboot the Oracle VM Server for the change to take effect. Manually editing the global Xen configuration parameters on an Oracle VM Server is not supported by Oracle.

It is possible to override the system-wide behavior by setting this parameter directly in the

./vmpinfo3.sh --username=admin listservers

28 for each individual virtual machine. This is the preferred approach to generating dump files, as it allows you to limit core dumps to only those virtual machines that you are interested in debugging. Therefore, this configuration option can be controlled for each virtual machine from within Oracle VM Manager. You can set this option by configuring the

./vmpinfo3.sh --username=admin listservers

42 option for a virtual machine. See the Servers and VMs Tab section in the Oracle VM Manager Online Help for more information on this parameter.

If you change the

./vmpinfo3.sh --username=admin listservers

42 option for a virtual machine, you must stop the virtual machine and then start it again before the change takes effect. This is different to restarting the virtual machine, as the

./vmpinfo3.sh --username=admin listservers

28 configuration file for the virtual machine is only read by the Xen hypervisor when the virtual machine is started. If you have made the configuration change but have not properly restarted the virtual machine, a crash and reboot does not automatically cause the configuration option to take effect.

To test whether or not the core dump facility is working properly for a virtual machine, you may be able to directly trigger a crash by logging into the virtual machine and obtaining root privileges before issuing the following command:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

3

This command assumes that the operating system on the virtual machine is Linux-based and that the System Request trigger is enabled within the kernel. After you have triggered the crash, check

./vmpinfo3.sh --username=admin listservers

39 on the Oracle VM Server where the virtual machine was running to view the dump file.

13.4.14 Tuning a Linux-based Virtual Machine for Handling Storage Migration

When a virtual machine is hosted in a repository using local storage on the Oracle VM Server where it is running, migration of that virtual machine to another Oracle VM Server and repository requires that I/O on affected disks is not excessively high. If you are running an application that has high I/O during a migration, it may cause the guest or the application to hang. Steps can be taken to mitigate against this on guests that are running a Linux operating system by tuning virtual memory caching parameters within the guest kernel and by reducing the ext4 journaling commit frequency on any guests that may be running file sytems that are formatted with ext4.

Tuning virtual memory caching. On the guest command line, as the root user, you can tune the cache by using the sysctl command to set a number of kernel parameters. Oracle recommends that you reduce the cache size to 5% of the system memory (the default value is 10) and reduce the time that a memory page can remain dirty until it is flushed to around 20 seconds (the default is 30 seconds). You can do this temporarily by running the following commands:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

4

Alternatively edit

./vmpinfo3.sh --username=admin listservers

13 and add these lines:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

5

When you have done this, you can load these values into the kernel by running sysctl -p.

Tuning ext4 journaling. If the guest is using any filesystems that are formatted to use ext4, the journaling commit process may be affected by a migration. To protect against this, decrease the amount of time between journal commits and ensure that commits are performed asynchronously. To do this, you should tune your mount parameters for any ext4 filesystem that you have mounted within the guest. For example when mounting an ext4 formatted filesystem you might use the following options:

Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

6

To perform this effectively for all ext4 mounts, you may need to edit your

./vmpinfo3.sh --username=admin listservers

47.