Oracle Labs by Yuri Khazin, Oracle DBA

April 14, 2016

RHEL 6.7, OEL 6.7 network adapters configuration and VM cloning


Cloning of a virtual machine in Oracle’s VB (Virtual Box) presents a particular pain. The exact clone, of course, is created with same MAC and IP addresses for all NICs present. Great for some purposes, not so great if you wanted a duplicate to repurpose. Then you would have reinitialized MACs during or after cloning and your trouble would start as all NICs in the clone won’t be functional upon power up. In older Linux distribution all you had to do is edit MAC addresses in the ifcfg-eth* files in /etc/sysconfig/network-scripts and that was it. Not so now.

In Linux 6 (Red Hat, Fedora, Centos) the hardware device control is primarily done by udev system and by NetworkManager service. This is an out of the box configuration. In the following exercise we will look at a case of a virtualized OEL 6.7 Server, which is a non-GUI Oracle’s package based on RHEL 6.7

Why Server edition? Because it is a headless, non-GUI distro, where all management needs to be done via command line and not through the nice desktop tools.

As stated above, in Linux 6 the udev system looks after changes in hardware and enumerates new NICs while preserving the old NIC records. Both parts are now present in this file: /etc/udev/rules.d/70-persistent-net.rules

Therefore, your /etc/sysconfig/network-scripts/ifcfg-eth* files are now not in sync with persistent rules. We will look at how they should be reconciled.

A few additional notes:

  • In my practice with OEL 6.7 the system-config-network-tui tool had no effect on configuration so I don’t recommend using it
  • Don’t just edit /etc/sysconfig/network-scripts/ifcfg-eth* files after cloning, read this article to the end to understand why you need to examine and change the udev part
  • OEL 6.7 Server (and probably Red Hat, Centos and Fedora distros) come with sshd server preconfigured with some annoying security features that make ssh login very slow. If you went through this article and still have issues logging into your machine you may need to read this guide about “Fixing SSH login long delay”

So, let’s start:

On original machine our network configuration is as follows:

eth0 is an adapter with static IP connected to host-only network

eth1 is an adapter with DHCP connected to NAT network

Both are running

Following are their respective config files

[root@oms1 network-scripts]# cat ifcfg-eth0
DEVICE=eth0BOOTPROTO=none

ONBOOT=yes

IPADDR=20.20.10.21

NETMASK=255.255.255.0

[root@oms1 network-scripts]# cat ifcfg-eth1
DEVICE=eth1HWADDR=08:00:27:F9:3C:34

TYPE=Ethernet

ONBOOT=yes

NM_CONTROLLED=yes

BOOTPROTO=dhcp

Notice, that although ifcfg-eth1 contains HWADDR keyword it is not required and can be removed when udev is allowed to manage the hardware.

Below is the contents of persistent rules (before cloning). Sometimes after cloning, the order of devices may be mixed up, so pay close attention to MAC addresses and then assign the desired names in proper order.

/etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules# program, run by the persistent-net-generator.rules rules file.

#

# You can modify it, as long as you keep each rule on a single

# line, and change only the value of the NAME= key.

# PCI device 0x8086:0x100f (e1000)

SUBSYSTEM==”net”, ACTION==”add”, DRIVERS==”?*”, ATTR{address}==”08:00:27:7d:f0:88″, ATTR{type}==”1″, KERNEL==”eth*”, NAME=”eth0″

# PCI device 0x8086:0x100e (e1000)

SUBSYSTEM==”net”, ACTION==”add”, DRIVERS==”?*”, ATTR{address}==”08:00:27:f9:3c:34″, ATTR{type}==”1″, KERNEL==”eth*”, NAME=”eth1″

After cloning with MAC address re-initialized the machine comes up like this (new NICs detected but since there are no matching ifcfg-eth* files the NICs are inactive):

Old NICs are eth0 and eth1, the new ones are eth2 and eth3

Delete old NICs:

Rename the ordering of NICs:

Make sure ifcfg* files match (optionally change the static IP)

Make sure original machine is NOT online as the clone will try to come up with same static IP. Reboot the clone machine (necessary, because of change to persistent rules)

After reboot of the clone:

If you want to change the static IP you can do it in ifcfg-eth0 and then restart networks

service network restart

Acknowledgements and references:

Advertisements

December 30, 2014

Building a Virtual Oracle RAC – Project Introduction


This article was originally written in 2010 and then updated several times. In 2010 the Virtual Box belonged to Sun Microsystems, now it is a part of Oracle’s toolbox. The interface of Virtual Box has somewhat changed over the years, so my apologies to the reader for screen snapshots that don’t look exactly like today’s version. You can still download older version of the program so that chances are that you won’t encounter issues while trying out this guide.

Introduction: This learning experiment is about building a totally virtual Oracle RAC environment at home using only open source software available to wide public (With the exclusion of Windows, if you happen to use it. I wish there was an open source version of MS Windows, that would benefit MS as well. Works for Red Hat, right?).

My project is generally based on an excellent article by Jeffrey Hunter (and other authors) – Build Your Own Oracle RAC Cluster on Oracle Enterprise Linux and iSCSI

Jeffrey’s article gives instructions on how to build a physical setup. I do recommend reading this article or at least giving it some 10 minutes scan before you attempt to build a RAC. In that article Jeffrey estimates a cost to build a production RAC in physical environment between $20K and $30K. He shows that for the purposes of learning the technology a “mostly physical” RAC with some parts virtualized can be built for under $2700. I am going to take the virtualization to the extreme and build a RAC inside a single PC. Yes, I do not care now about performance, and yes, I am willing to buy some additional memory for the PC. You can not really expect to run two Linux nodes with databases and a storage (SAN) simulation on a PC with 1GB of memory. My estimated cost is, therefore, somewhere under $100, if all we need is a memory upgrade. Sounds good? It is still a lot of work and it will probably take you many hours to do it right. The benefit is a hands on knowledge, which is priceless.

Now to the choice of hardware and software.

  • PC or server with some 200GB space and 3.5GB memory (XP Professional can handle up to 4GB).
  • The host OS: Windows XP SP3 professional (although SP2 will probably do as well)
  • Has been tested on Windows 7 Home and shown no issues
  • Can work on Linux host machine (Ubuntu, Debian, openSUSE, Fedora etc)
  • Virtualization software: Virtual Box 3.1.2 (or higher). Tested and works with no issues on both Intel and AMD processors (may need to adjust some CPU parameters in configuration).
  • Database nodes: Oracle Enterprise Linux 5 update 3 as a guest OS, running Oracle Database 10.2.0.1
  • SAN/NAS emulation software: Openfiler 2.3

My reasons for choosing particular software:

Host OS does not really matter, virtualization software can run on many different platforms. If you desire so, you can run this whole project using Linux as a host. Virtual Box is selected because it is an open source software produced by Oracle.  Virtual Box at 3.1.2 was still owned by Sun Microsystems, who acquired it from Innotek and then maintained from 2008. Oracle maintains this product from 2010.

Why Virtual Box and not VMware? VMware is a popular product but I do not see a consistency in their support of free versions of the software. In my experiments with VMware I have encountered some obstacles while running Linux guests (particularly in networking and CPU clock synchronization). Virtual Box is an open source product and quite a mature one, well documented and supported. So far I was able to run OEL4 and OEL5 with no issues (even with no guest additions installed). I like the product’s built-in networking, it seems to be more flexible and stable. These are all my personal preferences, of course.

Why Oracle’s Linux? This particular flavor of Linux is a modified Red Hat Enterprise Linux 5. Oracle has made a few adjustments that allow Oracle 10g R2 database (32 bit) to install with no issues on it (usually, no additional RPM’s required). Since I am aiming to install Oracle in the guest(s) thus the choice of this Linux flavor. Besides, this Linux comes with OCFS2 and most of the ASMLib software packages installed. If you have a licensing question about Oracle Enterprise Linux (OEL) I can answer it. You only need a license if you want Oracle’s support. For learning purposes you do not need a license. If you will need to download some rpm libraries and updates you can get those for free, the YUM configuration can be pointed to a free repository (not discussed here).

Now the database itself. Which edition of 10.2.0.1 are we going to use? There is no clarity in that article on this matter, for now I assume the Standard Edition will do. There are some additional pieces of software from Oracle (or used by Oracle) to be installed, those are: OCFS2, ASMLib 2.0 and TAF.

Now that we know what we need to get started let’s split the work into more manageable mini projects, follow them just in this order, and hopefully all pieces will fit:

Update (2015). There seems to be a viable alternative to Openfiler from the FreeNAS. First off, it is Free. Second, it is open source (or, maybe that was first). Then, they give you full documentation. There was time when we were on our own with Openfiler, as they wanted money for the tech documentation.

March 13, 2013

OEL 6.2 on Virtual Box 4.2.4

Filed under: Oracle VirtualBox, Virtualization, Windows 7 — Tags: , , — oraclelabs @ 10:28

I was installing “Oracle Linux Release 6 Update 2 Media Pack v1 for x86_64 (64 bit)” on VBox 4.2.4 the other day. It was on Windows 7 Home Premium SP1, the RAM given to the guest was set to 2GB (quite generous). The installation of “basic” server went well and by the end of it the guest wanted to reboot. This is when the dreaded message “Virtualbox Manager has stopped working” appeared. A research on forums was educational, but useless, as a wide variety of causes may be in play. Tried this and that, upgraded first to VBox 4.2.6, then to 4.2.8 with no luck. Then, remembering that this Linux version used to work well on older VBox I have downgraded to 4.1.22 and recreated host only networks (those were deleted during the reinstall). Guess what? All works fine now. Maybe in later versions of 4.2.x this problem will be resolved.

Blog at WordPress.com.