http://www.01informatique.fr/applications-115/pgi-cloud-risque-enfermement-52989/

A little precision : redundancy is on its way, so the post is a little outdated

This category’s goal is to explain by example how to use NiftyName capacities through its API.

You can download attached files at the end of the article and test at home

Today case study is

“how can I quick create my 5 servers cluster for my new website frontend ?”

You have zish and zigui of course, but sometimes it’s not pretty for repetitive tasks. Well, we need to make our own script. This script is written in Python. You write yours in any language, in future posts we’ll try to make some examples in PhP, Ruby, etc…

And… voilà ! the main script code is inlined behind. In downloadable archive there is some extra code, mainly code already used for zinn XMLRPC functionnal tests (if you’re curious, you can look at it).

To make this code work for you, customize the MyXmlRPC class and that’s all.

import os, sys, basexmlrpc

#Personnal XML-RPC class
class MyXmlRPC(basexmlrpc.BaseXmlRPC):
    def __init__(self):
        super(MyXmlRPC, self).__init__()
        self.confdir    = '%s/.zinn/users' % os.getenv('HOME')
        self.publicname = 'I am a nifty user'
        self.name       = 'niftyuser'
        self.passphrase = 'jablonski'
        self.sitegroup  = 'marseille'
        self.client = 'niftycli'
        self.vsite = 'web-frontend-vsite'
        #self.vsite = 'myniftyvsite'
        self.proxy = None

#Initialize connection
try:
    client = MyXmlRPC()
    client.proxy = client.get_authentified_proxy()
except Exception, e:
    print e
    sys.exit(0)

#Common attributes
how_many_frontends = 5
vsite_attributes = {'siteGroup': client.sitegroup, 'client': client.client}
machine_attributes = {
        'vcpu'  : 2,
        'cpu'   : 'x86-64bits',
        'memory': '2G',
        'vnc'   : {'keymap': 'en-us', 'password': 'Mys3scr3tp4sS'}
     }
storage_attributes = {'size': '3G'}
interface_attributes = {'model': 'virtio'}
attach_stor_attributes = {
            'bus'       : 'virtio',
            'media'     : 'disk',
            'readOnly'  : False,
            'bootOrder' : 1,
            'bootable'  : False
        }

#Bulk creation
client.proxy.vsite.create(client.vsite, vsite_attributes)

for i in range(how_many_frontends):
    client.proxy.machine.create(client.vsite, 'front%d' %i, machine_attributes)
    client.proxy.storage.create(client.vsite, 'stor%d' %i, storage_attributes)
    client.proxy.interface.create(client.vsite, 'iface%d' %i, interface_attributes)

    ipaddress_attributes = {'reverse': 'front%d.%s' %(i, client.vsite), 'family': 'ipv4'}
    client.proxy.ipaddress.create(client.vsite, 'ip%d' %i, ipaddress_attributes)

    client.proxy.interface.ipaddress.attach(client.vsite, 'iface%d' %i, 'ip%d' %i)
    client.proxy.machine.interface.attach(client.vsite, 'front%d' %i, 'iface%d' %i)

    status = 'notrunning'
    while status != 'running':
        r = client.proxy.storage.list(client.vsite, 'stor%d' %i)
        status = r[0]['status']['current']

    client.proxy.machine.storage.attach(client.vsite, 'front%d' %i, 'stor%d' %i, attach_stor_attributes)

This first sample lacks a lot of advanced functionnalities, like having all the cluster in a private network. OS installation is not covered too. Stay tuned, there’s more to come.

Download the archive here

Introduction

We performed all the tests with the multi-threaded benchmark tool sysbench (version 0.4.8).

http://sysbench.sourceforge.net/

This tool was primarily written for MySQL benchs and is very I/O intensive. Nowadays it’s one of the most appreciated tool for general I/O benchs.

Read bench commands

sysbench –test=fileio –max-time=60 –max-requests=100000000 –file-num=1 –file-extra-flags=direct –file-fsync-freq=0 –file-total-size=128M –file-test-mode=rndrd cleanup

sysbench –test=fileio –max-time=60 –max-requests=100000000 –file-num=1 –file-extra-flags=direct –file-fsync-freq=0 –file-total-size=128M –file-test-mode=rndrd prepare

sysbench –test=fileio –max-time=60 –max-requests=100000000 –file-num=1 –file-extra-flags=direct –file-fsync-freq=0 –file-total-size=128M –file-test-mode=rndrd run

Write bench commands

sysbench –test=fileio –max-time=60 –max-requests=100000000  –file-num=1 –file-extra-flags=direct –file-fsync-freq=0  –file-total-size=128M –file-test-mode=rndwr cleanup

sysbench –test=fileio –max-time=60 –max-requests=100000000  –file-num=1 –file-extra-flags=direct –file-fsync-freq=0  –file-total-size=128M –file-test-mode=rndwr prepare

sysbench –test=fileio –max-time=60 –max-requests=100000000  –file-num=1 –file-extra-flags=direct –file-fsync-freq=0  –file-total-size=128M –file-test-mode=rndwr run

Forces in presence

The goal is to be the most objective as possible, so we choose the closest configuration on each instance provider.

Flexiscale

ubuntu amd64 8.04 LTS

• 1 GiB RAM
• 20 GiB of storage

Gogrid

Centos amd64 5.1

• 1 GiB RAM
• database-server instance

EC2

ubuntu amd64 9.04

• small instance
• 1.7 GiB RAM

NiftyName

ubuntu amd64 9.04

• 1 GiB RAM
• 50 GiB of storage

Read

Transfer MB/sec : we are twice fast as the fastest competitor !

EC2, which is the most popular offer, suffers of very poor read performances.

I/O requests per second : it matchs read transfer values

Write

Transfer MB/sec : EC2 is a (slighty) more serious competitor

I/O requests/sec

Conclusion

NiftyName supremacy is undeniable. NiftyName does not forget performances against features. What is done is well done, we are trying to reach the highest quality level, it’s significant for a common production environment.

Facing the emergence of non-free platforms like Amazon EC2, Microsoft Azure and others it was necessary to bring a free solution. It’s the first early preview of our NiftyName. At the moment you can create and manage virtual machines pools and all related components (storage, private and public network, etc).

Multi-site active redundancy is not supported yet but all components have been designed for this.

The whole architecture is based on documented XMLRPC (SSL) webservices: you can make your own derivative tools on top.

Furthermore, two clients applications (console and GTK) on top of the API are already distributed.

This 1.0.0 verison supports the following functionnalities:

• Virtual machines (based on KVM, multi cpu, x86-64, VNC, etc)
• Storage (private, public, clone, shared between instances)
• Network (public addresses, IPv6 support, private Vlan, multiple interfaces)
• Users/clients management, permissions, roles, etc

A test platform is available, you can create test users and services today in real conditions.

We’re looking forward to read your remarks, propositions and contributions to make the project better and carry it to a higher level.