Ubuntu Enterprise Cloud: Experiencing the "Cloud" #2

Continue from last post, after solving the booting problem of vm instance (The cause of this problem is just my silly mistake which i ask vm to boot from a kernel image instead of vm image); several observations are obtained.


Observation-1
---
If you write to the root filesys of vm instance, the wrote data will not be saved to WSC when the instance is terminated. But, euca2ools provides "euca-bundle-vol" utility to "upload" a local filesys of an instance to WSC. That's to say ... u have to create another root filesys copy on WSC to save your write.


Observation-2
---
If a volume attached to 1 vm instance, it cannot be attached to other at the same moment unless it is detached. So, if you wanna host a shared data pool on eucalyptus, you have to use several vm instances to host nosqldb like mongodb or cassandra. And each of these instances has dedicated volume attached. Save your data via NoSQL :)


Observation-3
---
With eucalyptus managed network setting, network access to vm instances is controlled by security group. A security group will maintain a set of in-bound rules likes below:

PERMISSION admin default ALLOWS tcp 22 22 FROM CIDR 144.214.0.0/16
PERMISSION admin default ALLOWS tcp 22 22 FROM CIDR 10.2.0.0/16
PERMISSION admin default ALLOWS icmp 0 0 FROM CIDR 10.2.0.0/16
PERMISSION admin default ALLOWS icmp 0 0 FROM CIDR 144.214.0.0/16

For out-bound rules, setup firewall within the vm instances. Eucalyptus does not manage that.


Observation-4
---
As with eucalyptus managed private network, vm instances may use private IP address. To access them, you have to be connected to an instance at first and use the private IP address as locator. That's to say, you need to have at least 1 public IP address that the outside world can connect to an instance.

If you configured to use vlan enabled managed network, vm instances of different security group will have different subnet assigned. The virtual network isolation is done by this feature. To allow two subnet to communicate, add in-bound rules to the security group.

(But i'm still trying the network config, may have update on this later)



Personal opinion in managing the cloud with client tool
---
Hybridfox is great! But euca2ools with just cli is simple and even more great. I personally prefer euca2ools.


What's next??
---
Go ahead to host private AppEngine - appsacle. But, don't know whether i can get it managed with such a limited "availability" zone

Ubuntu Enterprise Cloud: Experiencing the "Cloud" #1

Continue from last post, this post documents my experience in setting up the ubuntu cloud.

Here is the resources i used to conduct the experiment:

* machine1 [Physical] - PentiumD 3GHz (core x2, VT-enabled), 2Gb RAM (512Mb x4), 200Gb HDD, NIC x1
* machine2 [vSphere VM] - Xeon X5560 2.8GHz (core x4), 2GB RAM, 100Gb HDD, NIC x2
* machine3 [vSphere VM] - Xeon X5560 2.8GHz (core x4), 2GB RAM, 200Gb HDD, NIC x1
* USB Thumb 1Gb x1
* CR-RW x2

To mostly align the architecture, the role of machines follow.
* machine1 - NC
* machine2 - CLC, WSC
* machine3 - CC, SC

And, these machines are connected by a single subnet (private network) 10.1.0.x while machine2 with another NIC connected to a "public" network.

Setup of the machines "strictly" follows the user guide except the setup of NTP server. A public IP address is provided to CC as the "elastic" address of VM instances. The availability zone is resulted as follow.



Each row of the table describes a particular type of VM that can be created. The availability of particular type can be found in the "free/max" column. The "max" is computed according to the number of CPU in NC machines by default (in this case, exactly 2). The number of CPU in NC can be shrank or grew according to config (see edit 2010-07-19). The cpu, ram, disk of particular type can be configured via the web interface of CLC.

After the installation of controllers, it's time to prepare the VM image. VM image has to be prepared by user and uploaded to WSC. The preparation requires KVM in UEC. If you don't wanna prepare your own image, just download it from ubuntu as shown.



After uploading the kernel, initrd ramdisk, and root file system (the vm image) to WSC, that's the time to use Hybridfox to start VM instances. Selects the uploaded image and launches VM instances according to that image.



When NC receives the request to launch VM instances, it retrieves the VM image from WSC. This is the "pending" phase of VM instance. Once image is loaded, VM instance gets booted and entering "running" phase. When VM instance receives shutdown request, it enters "shutting down" phase and finally goes to "terminated" phase. Phase change follows.



In this experiment, i try to launch 2 instances so that there will not be enough "elastic" IP addresses for the instances. My observation is that when launching the first instance, it acquires the "elastic" IP address. When launching the second instance, a complain is shown and asking to launch the instance with "private" address. Indeed, Eucalyptus CC contains a DHCP server and manages VLAN for "private" address. The "elastic" IP address can be detached from and attached to any instances at any time frame.

Let's get back to the last figure showing Hybridfox.



As you may noticed, the console output of the VM instance in the figure shows error message during the boot time. That's my next problem need to be solved.

***EDIT 2010-07-17*** The boot problem is solved... the cause is that i made a mistake to ask a VM to boot from a kernel image, instead of a vm image :P Now, i can ssh to the vm instance ^^"

***EDIT 2010-07-19*** The max of "Free/Max" of a node can be configured by NC's config MAX_CORES, MAX_MEM, and SWAP_SIZE, according to this post.

***EDIT 2010-07-20*** For MAX_MEM configured more than actual physical memory, sorry. See this post.

---

Something happened during the setup...

I was trying to burn ubuntu-1004-amd64 to one of the CD-RW. When using the CD-RW to install ubuntu onto the physical machine, it blamed that the disc was corrupted. Then, i tried burning another CD-RW... same corruption happened. Lastly, i used USB thumb as live-usb and got no file corruption.... As a result, i spent about 2 hours in doing this.

This is the first time i managed a machine with 2 legs (NIC x2) ... Some mis-configurations existed and thus slowing down the entire experiment.

Ubuntu Enterprise Cloud: Explaining the "Cloud"

Ubuntu Enterprise Cloud (UEC) depends heavily on KVM as the hypervisor and Eucalyptus as the elastic cloud solution.

In this post, a brief explanation of the Eucalyptus solution will be given.
[Disclosure: I just read a conference paper from Eucalyptus and a user guide to write this post... Some info. may not be detailed or having mistake. If there's any mistake, please point it out directly. I will later setup a private cloud for testing soon.]

Here is the architecture of Eucalyptus (direct linking from user guide).


There are few components in the architecture:

1. Cloud Controller, CLC (Interface with user)
2. Cluster Controller, CC (Sits in between CLC and NC, governing a cluster of node)
   
3. Node Controller, NC (Live in a node)
4. Walrus Storage Controller, WS3 (Keeping VM's kernel, root filesystem, and ramdisk)
5. Storage Controller, SC (The datastore)
   

Indeed, the very basic setup of UEC requires two machine. One of them MUST have Intel-VT / AMD-V enabled CPU for hardware virtualization acceleration (requirement of KVM indeed). So, let's say the first machine without Intel-VT / AMD-V CPU is named "uec-master" while another machine with the CPU is named "uec-node".

The Node Controller is going to be installed in the machine uec-node. NC is a software package that communicates with the KVM installed in uec-node. The communication is carried via libvirt. The "elastic" VM instances are going to be deployed onto uec-node running on top of KVM.

Other four controller: CLC, CC, WSC, SC can be installed on another machine uec-master. CLC is the software package that interfaces with user. CC is the package that masters a set of nodes (talking to NC directly for operations). WSC is the package to simluate Amazon S3 and maintaining the VM instance kernel, root filesys, and ramdisk. SC is the package to manage the actual datastore (volume or file space to be mounted) used by VM instances.

To setup VM instances, user have to first prepare the VM kernel and root filesystem (there're tools existed to aid you). This preparation is done via KVM. That's to say client machine used to prepare VM image would probably have Intel-VT/AMD-V CPU. After packaging the kernel and root fs, user can "upload" the package via CLC to WSC.

When user want to allocate resource for the VM instances, user have to assign a datastore for the instances. The datastore will be kept in SC. Once prepared, user issues instance-start to CLC and the CLC will forward the request to CC. CC will pick NC to serve the request; NC will finally load the the VM image from WSC and mount the volume from SC.

Thus, there will be 1 or more instances sharing the same volume from SC. The data persistence uses AoE or iSCSI protocol (which i have no idea at all yet :P).

So where does "elastic" come from? VM instances (CPU and memory resource) can be added to/removed from the cloud dynamically. SO elastic, man~ Apps running on VM instances have no idea of the CPU, memory, and the actual datastore. SO virtual, man~

Note that ... "any" Amazon S3 and EC2 client application would work with Eucalyptus as they share the same SOAP interface (REST interface for datastore).

Questions?

*** EDIT 2010-07-17 *** When a volume is attached to 1 vm instance, it cannot be attached to other vm instances at the same moment.

Quicksilver String Ranking Java Port

I search the phrase "Quicksilver String Ranking Java Port" in google... cannot find some interesting result...

so i just wrote one - http://github.com/mrkschan/qs-score-java/blob/master/QSString.java

One point to note is that ... In Java (v6)... String is a final class that cannot be extended... so i have to wrap a string and make this class less convenient.

Rationale for this: i'm writing a string filter for a eclipse plug-in which i wanna use a fast and excellent string ranking algorithm. Quicksilver is the way to go :)

香港紅十字會的來信

內容: 請繼續支持捐血救人 :) 下為你新的捐血卡 :P



其實睇下張舊卡, 就知道原來我兩年先捐得五次... 數量什少 =.=... (男性一年可以捐四次的呢)



玩開 FB 的朋友, 可以加 此App 留意紅十字會最近消息

Given a sequence of increasing values, find ...

The rationale of this post is hoping to find someone give me hints on a problem: Given a sequence of increasing values, find the first value which has a significant change in the sequence.

To visualize the problem: consider this sequence [1, 1, 2, 4, 6, 14, 24]
the "value" that has a significant change may be 14.

This can be plotted in a chart.

The slope in between value 6 and 14 gives a significant change, obviously.

Indeed, this is a problem that i encountered when implementing the DBScan algorithm.

DBScan algorithm is a "density based" clustering algorithm. It takes two parameters - radius of a circle (eps), and the number of points in the circle which the circle can be considered as dense (minPts).

You may think DBScan as simple as... Taking any point, a "sphere" is formed with the radius (eps) center at that point. The circle may cover other points in a N-dimensions space. If the number of points covered by the "sphere" is larger than minPts, it is dense enough to form a cluster. And that cluster further expands with the points covered by that "sphere".

Finally ... points that are "closely" distanced will form a cluster and all points in the N-dimension space will form "many" cluster.

An important parameter in DBScan algorithm is the radius. From the wikipedia, the radius can be estimated by k-distance graph. First, let's define what is k-distance. Given a point, k-distance is distance between the k-th closest point of the given point. A k-distance graph is plotted by getting k-distance of all points in the N-dimension space, sorting them and plot it out as shown above. From the graph, it is simple to get the sharp change k-distance as the eps and using k as the minPts.

But, my concern is raised since it's not practical to plot 100 graph and read them all if i wanna try out k from 1 to 101. As consequence, i wanna have a algorithm to get the k-distance without looking into a visual graph.

One idea flow to my mind is to compare the slopes of neighbouring k-distance. If k-distance of pt-b is much larger than that of pt-a (the point just before pt-b), then k-distance of pt-b may probably be the selected sharp change k-distance. However, what is "much larger than" in this method? A parameter of floating point? Thus, i think this method is not that practical.

Another idea is to use the statistics - mean and standard deviation. Mean is the average while standard deviation is the square root of the variance. Variance is the average difference from the mean value of every point. So, standard deviation split the k-distance graph piece by piece fairly. If this is the case, I can compute the mean and make use of standard deviation to obtain a k-distance in several split away from the mean. By this method, the parameter is the number of split away from the mean value. An integer that can loop nicely :]

My implementation of this method is hosted on github #line23. Anyway, i believe there must be better method to obtain k-distance "automatically"... please comment :]

Future Chromium bundles Flash ... Why.

There's a breaking news this morning from google's chromium blog - Bringing improved support for Adobe Flash Player to Google Chrome

In the post ... it mentioned..

We believe this initiative will help our users in the following ways:


* When users download Chrome, they will also receive the latest version of Adobe Flash Player. There will be no need to install Flash Player separately.

* Users will automatically receive updates related to Flash Player using Google Chrome’s auto-update mechanism. This eliminates the need to manually download separate updates and reduces the security risk of using outdated versions.

* With Adobe's help, we plan to further protect users by extending Chrome's “sandbox” to web pages with Flash content.

The 1st two points are trivial, but the 3rd? This is the reason why i'm writing this post.

The 3rd generated my interest in knowing more about the sandbox model of Chrome and thus leading me to read their "technical report".

In the following paragraph, i will briefly explain the architecture of Chromium based on reading their "technical report". (Note: I didn't read any source code of the beautiful Chromium project and their "report" was published quite a while ago. Thus, some info. below may be inaccurate or outdated but fairly enough for u to get understand with their sandbox model.)

---
Here we go.
---

In the chromium architecture, the web browser is separated into two core components: the rendering engine and the browser kernel.

Here is the task assignments mentioned in the report.

Rendering Engine: HTML parsing, CSS parsing, Image decoding, Javascript interpreter, Regular expressions, Layout, Document object model, Rendering, SVG, XML parsing, XSLT

Browser Kernel: Cookie database, History database, Password database, Window management, Location bar, Safe browsing blacklist, Network stack, SSL/TLS, Data cache, Download manager, Clipboard

And, both of the components can carry URL parsing and Unicode parsing.

From their design philosophy, the rendering engine is always subjected to a bunch of known / unknown vulnerabilities. Thus, they decided to grant limited privileges to the rendering engine while letting the browser kernel to have the user's privileges.

These two components are communicating with IPC (Inter-Process communication) from a "big" picture in the report and the browser kernel provides API for the rendering engine to use its services.

Before i continue, here is a remark. Their report mentioned:

Chromium's security architecture mitigates approximately 70% of critical browser vulnerabilities that let an attacker execute arbitrary code.

Where is the rest 30%? You will, perhaps, know later.

Whenever a user create a new tab in Chrome, the browser kernel fork() a new rendering engine with a sandbox. What is the sandbox? On windows, it's the Windows Security Manager. The rendering engine runs with a "restricted security token", that is used by the security manager, on a Windows Job Object. Whenever some action is performed by the rendering engine, the manager will check for privileges.

It sounds work ... but some known limitations in this M$ windows sandbox.
[1] FAT32 - FAT32 does not support the access control list and thus user's USB thumb with FAT32 format may be read or write by a compromised rendering engine.
[2] Misconfigured objects - Some application create objects with NULL DACLs. This will let security manager ignore the security token owned by rendering engine. (But NTFS mitigated that)
[3] TCP/TP - Theoretically, rendering engine can create any socket as it wishes... (But Win API require a "handle", which the rendering engine will not hv, to open a socket ... a bug in Win API?? or a feature??)

As a result of using the sandbox, even a rendering engine is compromised by buffer overflow of HTML parser, integer overflow of regex, or what-so-ever related to attack against rendering engine... [a] the attack can only use the restricted sandbox to do the bad ... [b] or invoke the IPC with Chrome's browser kernel ... [c] or if m$ security manager hv bug, sandbox broke too

You may noticed that [a] is assumed to be secured and [c] has nothing to do at all since m$ is "closed". The only problem left is [b] ... attacker insert code to invoke browser kernel to do bad.

Now, let's see how's kernel work.

Browser kernel API handles a few things. User interaction, persistent storage, and networking.

User interaction includes - rendering and user input.
Rendering is handled by the rendering engine. Unparsed HTML is sent to the engine and a bitmap is returned to the browser kernel. The kernel uses the trick like "double-buffering" for the bitmap and presents it onto screen.
User input either occurs on Chrome's body (they call it browser chrome) like location bar or within the web page. For former case, kernel will handle it. For latter case, kernel asks rendering engine to render the input as bitmap again. Even rendering engine is compromised, it cannot receives keyboard signals.

Persistent storage includes - upload and download.
Upload is handled by kernel. When user selects certain file, kernel will ask to render the selection path and keeps records of which file the user granted to the web (engine) to access. Even compromised, the engine cannot touch those files which are not granted yet.
Download is also handled by kernel. Kernel API provides method to invoke a download action which the kernel will blacklist some filename like desktop.ini (M$ vulnerable feature). Even a rendering engine is compromised, the engine cannot ask the kernel to download some file to anywhere on user's disk.

Networking includes URL requests.
Rendering engine itself is allowed to fire requests for protocol like http, https, ftp, etc. But not the file://
File opening is handled by browser kernel when user type in file:// in the location bar. The file is rendered in a dedicated rendering engine.

After the long boring things... you will noticed that ... Chrome actually deals with 3 security threats. [1] Persistent Malware, [2] Transient Keylogger, and [3] File Theft. Other threat like phishing, origin isolation, firewall circumvention, and website vulnerabilities are not in scope of Chromium written in the "technical report".

Last but not least ... why bundle Flash (a "the most widely used web browser" plugin?) There is a small paragraph in the report talks about plug-in. From historical reason, plug-in is a separate host process which execute outside the browser.

In order to maintain compatibility with existing web sites, browser plug-ins cannot be hosted inside the rendering engine because plug-in vendors expect there to be at most one instance of a plugin for the entire web browser. If plug-ins were hosted inside the browser kernel, a plug-in crash would be sufficient to crash the entire browser... each plug-in runs outside of the sandbox and with the user's privileges... For example, the Flash Player plug-in can access the user's microphone and webcam, as well as write to the user's file system (to update itself and store Flash cookies).

...

Chromium also contains an option to run existing plug-ins inside the sandbox. To do so, run the browser with the --safe-plugins command line option. This setting is experimental and might cause instability or unexpected behavior.

So... you know why "With Adobe's help, we plan to further protect users by extending Chrome's “sandbox” to web pages with Flash content." (Flash player like m$ is "closed")

The chromium team phoned adobe and says... let's work together to secure Flash and silent those who against Flash coz of security concern in unknown implementation flaws. H.264 video codec will keep on our Utube. (this paragraph is some "bs" that comes to my mind)