Ubuntu Enterprise Cloud: Experiencing the "Cloud" #2

Continue from last post, after solving the booting problem of vm instance (The cause of this problem is just my silly mistake which i ask vm to boot from a kernel image instead of vm image); several observations are obtained.


Observation-1
---
If you write to the root filesys of vm instance, the wrote data will not be saved to WSC when the instance is terminated. But, euca2ools provides "euca-bundle-vol" utility to "upload" a local filesys of an instance to WSC. That's to say ... u have to create another root filesys copy on WSC to save your write.


Observation-2
---
If a volume attached to 1 vm instance, it cannot be attached to other at the same moment unless it is detached. So, if you wanna host a shared data pool on eucalyptus, you have to use several vm instances to host nosqldb like mongodb or cassandra. And each of these instances has dedicated volume attached. Save your data via NoSQL :)


Observation-3
---
With eucalyptus managed network setting, network access to vm instances is controlled by security group. A security group will maintain a set of in-bound rules likes below:

PERMISSION admin default ALLOWS tcp 22 22 FROM CIDR 144.214.0.0/16
PERMISSION admin default ALLOWS tcp 22 22 FROM CIDR 10.2.0.0/16
PERMISSION admin default ALLOWS icmp 0 0 FROM CIDR 10.2.0.0/16
PERMISSION admin default ALLOWS icmp 0 0 FROM CIDR 144.214.0.0/16

For out-bound rules, setup firewall within the vm instances. Eucalyptus does not manage that.


Observation-4
---
As with eucalyptus managed private network, vm instances may use private IP address. To access them, you have to be connected to an instance at first and use the private IP address as locator. That's to say, you need to have at least 1 public IP address that the outside world can connect to an instance.

If you configured to use vlan enabled managed network, vm instances of different security group will have different subnet assigned. The virtual network isolation is done by this feature. To allow two subnet to communicate, add in-bound rules to the security group.

(But i'm still trying the network config, may have update on this later)



Personal opinion in managing the cloud with client tool
---
Hybridfox is great! But euca2ools with just cli is simple and even more great. I personally prefer euca2ools.


What's next??
---
Go ahead to host private AppEngine - appsacle. But, don't know whether i can get it managed with such a limited "availability" zone

Ubuntu Enterprise Cloud: Experiencing the "Cloud" #1

Continue from last post, this post documents my experience in setting up the ubuntu cloud.

Here is the resources i used to conduct the experiment:

* machine1 [Physical] - PentiumD 3GHz (core x2, VT-enabled), 2Gb RAM (512Mb x4), 200Gb HDD, NIC x1
* machine2 [vSphere VM] - Xeon X5560 2.8GHz (core x4), 2GB RAM, 100Gb HDD, NIC x2
* machine3 [vSphere VM] - Xeon X5560 2.8GHz (core x4), 2GB RAM, 200Gb HDD, NIC x1
* USB Thumb 1Gb x1
* CR-RW x2

To mostly align the architecture, the role of machines follow.
* machine1 - NC
* machine2 - CLC, WSC
* machine3 - CC, SC

And, these machines are connected by a single subnet (private network) 10.1.0.x while machine2 with another NIC connected to a "public" network.

Setup of the machines "strictly" follows the user guide except the setup of NTP server. A public IP address is provided to CC as the "elastic" address of VM instances. The availability zone is resulted as follow.



Each row of the table describes a particular type of VM that can be created. The availability of particular type can be found in the "free/max" column. The "max" is computed according to the number of CPU in NC machines by default (in this case, exactly 2). The number of CPU in NC can be shrank or grew according to config (see edit 2010-07-19). The cpu, ram, disk of particular type can be configured via the web interface of CLC.

After the installation of controllers, it's time to prepare the VM image. VM image has to be prepared by user and uploaded to WSC. The preparation requires KVM in UEC. If you don't wanna prepare your own image, just download it from ubuntu as shown.



After uploading the kernel, initrd ramdisk, and root file system (the vm image) to WSC, that's the time to use Hybridfox to start VM instances. Selects the uploaded image and launches VM instances according to that image.



When NC receives the request to launch VM instances, it retrieves the VM image from WSC. This is the "pending" phase of VM instance. Once image is loaded, VM instance gets booted and entering "running" phase. When VM instance receives shutdown request, it enters "shutting down" phase and finally goes to "terminated" phase. Phase change follows.



In this experiment, i try to launch 2 instances so that there will not be enough "elastic" IP addresses for the instances. My observation is that when launching the first instance, it acquires the "elastic" IP address. When launching the second instance, a complain is shown and asking to launch the instance with "private" address. Indeed, Eucalyptus CC contains a DHCP server and manages VLAN for "private" address. The "elastic" IP address can be detached from and attached to any instances at any time frame.

Let's get back to the last figure showing Hybridfox.



As you may noticed, the console output of the VM instance in the figure shows error message during the boot time. That's my next problem need to be solved.

***EDIT 2010-07-17*** The boot problem is solved... the cause is that i made a mistake to ask a VM to boot from a kernel image, instead of a vm image :P Now, i can ssh to the vm instance ^^"

***EDIT 2010-07-19*** The max of "Free/Max" of a node can be configured by NC's config MAX_CORES, MAX_MEM, and SWAP_SIZE, according to this post.

***EDIT 2010-07-20*** For MAX_MEM configured more than actual physical memory, sorry. See this post.

---

Something happened during the setup...

I was trying to burn ubuntu-1004-amd64 to one of the CD-RW. When using the CD-RW to install ubuntu onto the physical machine, it blamed that the disc was corrupted. Then, i tried burning another CD-RW... same corruption happened. Lastly, i used USB thumb as live-usb and got no file corruption.... As a result, i spent about 2 hours in doing this.

This is the first time i managed a machine with 2 legs (NIC x2) ... Some mis-configurations existed and thus slowing down the entire experiment.

Ubuntu Enterprise Cloud: Explaining the "Cloud"

Ubuntu Enterprise Cloud (UEC) depends heavily on KVM as the hypervisor and Eucalyptus as the elastic cloud solution.

In this post, a brief explanation of the Eucalyptus solution will be given.
[Disclosure: I just read a conference paper from Eucalyptus and a user guide to write this post... Some info. may not be detailed or having mistake. If there's any mistake, please point it out directly. I will later setup a private cloud for testing soon.]

Here is the architecture of Eucalyptus (direct linking from user guide).


There are few components in the architecture:

1. Cloud Controller, CLC (Interface with user)
2. Cluster Controller, CC (Sits in between CLC and NC, governing a cluster of node)
   
3. Node Controller, NC (Live in a node)
4. Walrus Storage Controller, WS3 (Keeping VM's kernel, root filesystem, and ramdisk)
5. Storage Controller, SC (The datastore)
   

Indeed, the very basic setup of UEC requires two machine. One of them MUST have Intel-VT / AMD-V enabled CPU for hardware virtualization acceleration (requirement of KVM indeed). So, let's say the first machine without Intel-VT / AMD-V CPU is named "uec-master" while another machine with the CPU is named "uec-node".

The Node Controller is going to be installed in the machine uec-node. NC is a software package that communicates with the KVM installed in uec-node. The communication is carried via libvirt. The "elastic" VM instances are going to be deployed onto uec-node running on top of KVM.

Other four controller: CLC, CC, WSC, SC can be installed on another machine uec-master. CLC is the software package that interfaces with user. CC is the package that masters a set of nodes (talking to NC directly for operations). WSC is the package to simluate Amazon S3 and maintaining the VM instance kernel, root filesys, and ramdisk. SC is the package to manage the actual datastore (volume or file space to be mounted) used by VM instances.

To setup VM instances, user have to first prepare the VM kernel and root filesystem (there're tools existed to aid you). This preparation is done via KVM. That's to say client machine used to prepare VM image would probably have Intel-VT/AMD-V CPU. After packaging the kernel and root fs, user can "upload" the package via CLC to WSC.

When user want to allocate resource for the VM instances, user have to assign a datastore for the instances. The datastore will be kept in SC. Once prepared, user issues instance-start to CLC and the CLC will forward the request to CC. CC will pick NC to serve the request; NC will finally load the the VM image from WSC and mount the volume from SC.

Thus, there will be 1 or more instances sharing the same volume from SC. The data persistence uses AoE or iSCSI protocol (which i have no idea at all yet :P).

So where does "elastic" come from? VM instances (CPU and memory resource) can be added to/removed from the cloud dynamically. SO elastic, man~ Apps running on VM instances have no idea of the CPU, memory, and the actual datastore. SO virtual, man~

Note that ... "any" Amazon S3 and EC2 client application would work with Eucalyptus as they share the same SOAP interface (REST interface for datastore).

Questions?

*** EDIT 2010-07-17 *** When a volume is attached to 1 vm instance, it cannot be attached to other vm instances at the same moment.