Everyone is familiar with the operation of installing and using xen virtual machines, but some are not very familiar with xen migrate operations. Friends who want to know can refer to it.
1. Preparation for migration
Before the migration, there is some preparation to be done, mainly the destination host and the source host and its network aspects. The following are introduced separately.
First, both must have Xen and xend daemons running in both source and destination hosts. It is necessary to ensure that the destination host has sufficient disk space, memory capacity and resources for the operation of the migrated domain. In addition, the source and destination hosts must also have the same architecture and virtualization extensions, for example, if the source host uses an x86-64 architecture with virtualization extensions, the destination hosts must also do so. This is done to prevent the domain from working properly after the migration due to mismatch between the instruction set used by the kernel and the user library.
Secondly, in terms of network, the source and destination hosts are required to be on the same subnet. When migrating a domain, if the destination node is on a different subnet, the migration will fail because the MAC and IP addresses of the domain are also migrated. If you run a firewall, you may need to create the corresponding iptables rules to allow migration to the connection.
Furthermore, during migration, the xend daemon aborts the domain's run on the source host, then copies it to the destination host, and restarts the domain. By default, the xend daemon accepts migration requests from the local machine. In order for the migration target to accept migration requests from the remote host, you must modify the xen-relocation-hosts-allow parameter of the destination host in the /etc/ file. Because no authentication is provided, for security reasons you must limit the hosts that allow migration.
Next, we begin to introduce the first method for migrating domains: storage restore.
2. Storage restore and migration method
The process of storing and restoring is as follows: First, save the current running state of a domain to a file on the hard disk, and then Xend uses this file to restore the original state of the domain. In fact, this is very similar to the hibernation function of a laptop computer: when sleeping, the laptop computer will save an image of the hard disk state and then shut down automatically. When exiting Hibernation, it uses the saved disk image to restore the system to its previous running state.
In this section, we introduce to readers how to save a running domain as a file, and then use it to restore the domain to its previous working state. As for where to restore it, it can be on the same server or on an outside Xen server. But before saving the domain as an image file, be sure to free up enough disk space to store the file.
Operation process
OK, let’s start by introducing the specific steps for migrating Domain on Xen server. What you need to do is create a Debian domain and save its status to a file, which will be used to restore the Debian domain.
1. Create a Debian guest domain:
# xm create debian_etch_domU.cfg –c
2. Check whether the domain is running normally. Since we provide the -c parameter to the xm command above, this will start the client Domain in the current console. However, to check whether the Domain has been started, another console session must be used.
3. We will save the current status of this Domain to the /xen-saved-images directory, and of course it can also be saved to other directories of your choice:
# xm save 1 /xen-saved-images/ The source virtual machine is turned off
4. The saved Domain will be deleted from the currently running Domain list. We can use the xm command to list the currently running Domain to see if it has been deleted.
5. Restore saved Domain from the file:
# xm restore /xen-saved-images/
6. Execute the xm command again and we will see that the restored Domain is back to the current Domain list.
Related Instructions
When saving the running status of Domain at work to a disk file, the file is not encrypted in any form, so malicious users can tamper with these images as long as they can access the directory where the image files are saved. Therefore, it is particularly important to ensure the security of access to folders where these images are stored.
The running state of a Domain is a snapshot or image of the Domain at the time it is saved. All running processes' information and status are stored in this file. When you look at the file, you will find that this is a huge binary file with a size equal to the amount of memory used when the Domain was saved. Because Xen Domain usually runs some enterprise applications, this file is generally small, usually around a few GB. Therefore, you must ensure that there is enough storage space to store this file. Although this example keeps the status on a hard disk file, you can also store the file on a large enough network folder, or even on a USB drive. It's really good to save Domain to USB drive, so you can carry it with you, and you can restore it on different servers if needed.
We can restore the stored Domain using the xm command plus the restore option, which will start the Domain again and restore it to its saved state. When using the storage restore method, there are some things that need to be paid attention to: because the ID restored by Domain is different from the Domain ID at its storage time, when using xm to view the restored Domain, you must pay attention to the changes in the ID. Although Domain has been restored to its original state (the state of being stored at the moment), it cannot be automatically connected to the console of the domain. Instead, we must explicitly run the xm command to connect to the console.
Although this way of migrating Domain is simple and easy, it is important to note that since Domain is deleted from the active Domain when saved, various services running on the Domain cannot be used or accessed. If you are unacceptable to such business outages, you should consider adopting the dynamic migration characteristics of Xen as described in the next section
3. Dynamic migration method
Above, we introduce the storage restore method when migrating Domain on Xen server. Although this approach is simple, if you have critical business on your server that is not allowed to be interrupted, or you want to minimize business interruption time, you still cannot use this migration method.
At this time, Xen also provides us with a powerful function that comes in handy, which is the dynamic migration method. It allows Domain to migrate Domain to another Xen server at the expense of minimal service interruption during operation.
Using Xen dynamic migration can be described as many benefits. First, the main advantages are listed as follows:
·Xen's dynamic migration works with high availability solutions such as heartbeat, which can give us a "never break down" system. The latest versions of Enterprise SuSe Linux Server and Red Hat Enterprise Linux also use Xen to provide a variety of high availability solutions. You can easily meet the demanding requirements of various services, while ensuring that all critical business services are not interrupted.
·It enables us to maintain the physical servers that house virtual machines in a "preventive" way. You can monitor the server and then transfer the system to resolve potential and suspicious issues instantly.
·It enables load balancing between multiple servers, allowing us to better utilize all the computing resources in the enterprise and make their utilization optimally. It should be noted that the open source version of Xen does not currently support the function of automatically performing dynamic migration when a failure is sensed on dom0.
·It makes it easier to add computing power to system configurations when needed.
·You can replace the hardware as needed without interrupting the services running on that hardware.
It is not enough to just know the benefits of dynamic migration. Now let me introduce the specific usage method.
Require:
- Both must run Xen and xend daemons
- It is necessary to ensure that the destination host has sufficient disk space, memory capacity and resources
- The source and destination host must also be on the same subnet
- The source and destination hosts must also have the same architecture and virtualization extensions
Configuration file:
[root@host1 ~]# vi /etc/xen/
(xend-relocation-server yes)
(xend-relocation-port 8002)
(xend-relocation-address '')
(xend-relocation-hosts- allow '')
Execute again: /etc//xend restart
Specific operations
First, let’s introduce the servers involved in dynamic migration, they are:
·hostA: A linux NFS server that provides storage space for XenDomain over the network.
·hostB: An Xen host server that runs a client using the NFS exported directory Domain: Ubuntu Fuwa.
·hostC: An Xen host server, which is the migration destination of the client Domain from the hostB server, that is, Ubuntu Fuwa.
The following figure shows us the specific configuration
⒈ Dynamic migration requires shared memory. In reality, except for the requirements set in the first section, the minimum requirement for network speed is 100MB. In order to share the memory, we will build an NFS server on the Xen host at the top of the image above. The Domain you want to migrate uses the shared memory space on the NFS server.
⒉ Edit the /etc /exports file and add the following line to export the memory directory:
# /xen-storage *(r，sync，no_root_squash)
⒊ Save the file and restart the NFS server. Add an NFS server to the server's service and set it to run on system restart:
# service nfs startchkconfig nfs on
⒋ We will build two Xen servers and let them use the memory provided by the NFS server. In order to mount the export directory, you need to add an installation point for both Xen servers:
# mount hostB:/xen-storage
#- mount hostC:/xen-storage
⒌ Modify the Xend configuration file to allow dynamic migration. Edit the /etc/xen/ file and make sure the following two lines are not commented out:
(xend-relocation-port 8002)
(xend-relocation-address '')
Its role is to start the Xen daemon to listen and respond to various requests for dynamic migration.
⒍ Create a Xen Domain running on NFS server memory and start the client Domain on hostB. And let the client Domain use the following configuration:
cat > /home/pchaganti/xen-images/ubuntu_fuwa_nfs_domU.cfg << "EOF"
kernel = "/boot/vmlinuz-2.6.16.38-xenU"
memory = 256
name = "ubuntu_fuwa_nfs_domU"
vif = [ 'ip=192.168.1.111' ]
nfs_server = '192.168.1.67'
nfs_root = '/xen-storage'
root = '/dev/nfs'
EOF
⒎ Now, we have a client Domain running on hostB, which is stored using the NFS exported directory. We can run the following command on hostB to check whether the Domain is running normally:
⒏ Next, we dynamically migrate this client Domain to the hostC server, which may take several minutes to do so. If the migration fails, a message indicating failure is displayed on the console.
# xm migrate --live ubuntu_fuwa_nfs_domU hostC
⒐ The above is all the work of dynamically migrating the running Domain to another Xen server! Now use the xm command to display the Domain currently running on the two servers, hostB and hostC. You will find that the migrated Domain can only be found on the hostC server. The shadow of the migrated Domain is no longer found in the Domain running on the hostB server listed.
Related Instructions
Dynamic migration is the process of moving a virtual machine from one physical host to another in a "powered" situation. This process will not have a significant impact on the end user, allowing administrators to perform offline repairs or upgrades to the physical server without affecting the normal use of the user.
It should be noted that in order to change the running location of Domain, the default Xend configuration file must be modified. In addition, for the modifications to take effect on the host, the host on which the Xen server is located must be restarted. We need to modify the following configuration:
·Xend-relocation-server: This flag is used to enable/cancel the server migration function. By default, this flag is set to no, which means that the server location is not allowed to be changed. During the migration process, Domain virtual memory will become the original form without any encryption. So be careful when enabling this option on an untrusted network.
·Xend-relocation-port: The xend daemon is migrated through this port. The default value of this port is 8002.
In addition, there are two parameters that do not need to be modified. Pay attention to when migrating in an enterprise deployment environment:
·Xend-relocation-address: This flag restricts Domain migration to specific interfaces. The specified address is the interface address that listens for incoming connections used for Domain migration. This flag will only be used after the xend-relocation-server parameter is enabled.
·Xend-relocation-hosts-allow: This flag defines the host that allows communication with the migrated port. Its value is a regular expression sequence separated by spaces. If the value is empty, all input connections will be allowed, otherwise, the value will either match an IP address or a complete domain name
In fact, the migration process from hostB to hostC is divided into multiple stages, as shown below:
Phase 1: Before the migration.
At this time, the Fuwa Domain runs on hostB and is an active Domain.
Phase 2: Booking.
Make a migration request on hostB and then check and confirm the resources available on hostC. Xend then prepares a VM container on hostC according to the desired size. If Xend can't get the resources you need on hostC, then it's all over. The Domain is still running on hostB, just as nothing happened, and this migration attempt was abandoned.
Stage 3: Repeat pre-copy.
First, transfer all memory pages from hostB to hostC, and then continue to transfer pages that have changed during the previous transfer. Eventually, the pages are copied to hostC.
Until the third phase, Fuwa Domain still operates normally on hostC.
Stage 4: Abort and copy.
Domain running on hostB is suspended and all its network traffic is redirected to hostC. After this stage ends, in addition to a suspended Domain on hostB, there is also a copy of this suspended Domain on hostB. At this time, the suspended Domain on hostB can also be used as the original copy. In case this step fails, it can also resume running on hostB.
Phase 5: Delivery.
If this step is not error or failure, hostC will send an indication to hostB stating that it already has a Domain image that is exactly the same as hostB. Now hostB will delete the Domain from its Xen server. From then on, hostC will become the first host of this Domain.
Fuwa Domain is in a downtime state during the fourth to fifth phases.
Stage 6: Activation.
Activate the migrated Domain on hostC. All device drivers inside the Domain are now associated to the new machine. At this point, Fuwa Domain is resurrected on hostC.
These steps are carried out behind the scenes. It is this series of unknown operations that enables the migration to transition so smoothly and seamlessly!
4. Conclusion
In this chapter, we study two ways to migrate the Xen client Domain: storage restore method and dynamic migration method. The former method is similar to the deep hibernation of a laptop computer. It first stores the captured domain running state on disk as an image, and then restores the domain using the image. The latter method dynamically changes the operating position of the domain in the running state with minimal interruption and downtime. Combining these two migration methods will surely make the effectiveness of virtualization technology a step further.