Build your own cloud – Tutorial

Make your own private cloud so you can access your files from any device, including PCs, smartphones and tablets, from anywhere in the world… 

(well, in the first place, this is a home based solution to personal storage on the cloud. If you have set up a web server that is accessible by the outside world, then you can access it anywhere from the world. - z.y. wang)

Cloud computing is more than a buzzword. It is an essential part of how we use technology today. On a personal level, we all store and retrieve data and we do this on multiple devices, like PCs, smartphones, tablets and media players. We also want a coherent experience with this data: we want to be able to retrieve and store any data from/on any device. In the old days, we had to manually sync every other device to a central location to have the same copy of data everywhere else. But this is a very cumbersome and confusing process. On top of that, you will also have to limit where data is being saved. In most of the cases this place is the central system from where we are syncing the other devices. This method is still not that bad if you are doing this between just two devices; but when you go beyond that, you will always think that there must be a better way to do this.
Imagine having all your data available to you on every device you own. You are not bound by limitations such as where you can store or retrieve the data. This is where the cloud comes into the picture and makes it all a reality. Cloud computing in a more general sense means making computing resources (like storage, processing power, software etc) available in the form of services (on public or private networks) which can then make those resources available from any other system.
In this feature we will be building our own cloud system, a cloud storage system to be precise. Much like Dropbox, Ubuntu One or SkyDrive, but our implementation will have more features. Before you move on, you may ask what is the point in doing this when we already have such popular services on our disposal. Here is why:
Not your cloud: While these services are popular and easy to use, you do not own them. You use them by agreeing to certain terms and conditions (we bet you haven’t even read them) that can be changed at any given time by the service providers. This may leave your files at risk.
Downtime: Sometimes you may lose access to your files because of server issues faced by your service provider. This can very problematic if you are unable to gain access to one of your important files when you really need it.
Privacy: Since your files are hosted elsewhere, they may be viewable to certain third-party agencies (like governments) without your consent. This worries a lot of people.
Features and restrictions: Most of these proprietary services do not have a lot of features and are very restrictive in terms of what you can do with the files.

The interface

Step by Step

We will be building our private cloud using the open source software called ownCloud. The name says it all: it allows you to build your own cloud storage infrastructure. OwnCloud started its life as a The KDE cloud computing project and is now available on almost all popular platforms. Created by KDE developer Frank Karlitschek, it is now developed by the ownCloud team.

Step 01

Installing the prerequisites
The ownCloud core is written in PHP5. Its prerequisites are basic, like those of any typical PHP web application. On the database side it supports SQLite, MySQL, Oracle and PostgreSQL. For our setup we’ll use MySQL to keep things simple yet scalable. If you are only doing a test setup, you can use SQLite, which is a zero configuration database system.
You will need to install the following on your Linux distribution:
PHP packages: php5, php5-gd, php-xml- parser,php5-intl
Database driver: php5-mysql. If you are planning to use any other database you will also need to install the respective database together with its PHP driver.
Curl packages: curl, libcurl3, php5-curl
SMB client: smbclient; this is used to mount Windows share.
Web server: apache2
For a Debian-based distribution, you can run the following command to install all the prerequisite packages:

$ sudo apt-get install apache2 php5 php5-gd php-xml-parser php5-intl php5-sqlite php5-mysql smbclient curl libcurl3 php5-curl mysql-server

Step 02

Installing ownCloud – setting up the web root directory
Download the latest version of ownCloud from http://owncloud.org/releases/. For this tutorial we are using owncloud-4.5.6.tar.bz2 At this point you’ll need to set up the web server root directory. On Debian-based distributions it is /var/www. Extract the owncloud package in the web server root directory When in doubt, look at the Apache configuration file.

$ tar xjf owncloud-4.5.6.tar.bz2
$ cp -r -v owncloud/ /var/www/ ludcloud //

In our example, we are using ludcloud as the root installation directory.
OwnCloud needs to write to certain directories of its installation. To do that, the web server user (www-data for Debian-based distributions) must own apps, data and config directories of the installation. Run the following commands to give the required permissions:

$ cd /var/www/ludcloud
$ sudo mkdir data //

This folder is not present, but is needed during installation.

$ sudo chown -R www-data:www-data data
$ sudo chown -R www-data:www-data config
$ sudo chown -R www-data:www-data apps

Step 03

Installing ownCloud – configuring the web server
In this step we will be configuring Apache Web Server for ownCloud. OwnCloud requires .htaccess to be enabled on the Apache server. .htaccess files (or ‘distributed configuration files’) provide a way to make configuration changes on a per-directory basis. To enable the .htaccess in the web server, edit your web server configuration file (in Debian-based distributions it is /etc/apache2/sites-enabled/000-default) with AllowOverride All.

   <Directory /var/www/>
        Options Indexes
FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
   </Directory>

Next we need to enable an Apache module called mod_rewrite. Mod_rewrite provides a rule-based rewriting engine to rewrite requested URLs on the fly. To enable this module, run the following commands:

$ sudo a2enmod rewrite
$ sudo a2enmod headers

Once you have enabled the necessary modules, you can restart the service to apply the changes.

$ service apache2 restart

Step 04

Installing ownCloud – running the installation
Navigate to http://localhost/ludcloud. You will be presented with the installation screen for ownCloud.
The first thing to do here is to create an admin account. Enter the desired username and password to do so. To configure the database, click on Advanced then select MySQL. Then enter the MySQL username password along with the database name. If you do not have a separate database user configured then you can use the root username; ownCloud will create a dedicated database with a dedicated db user for use with ownCloud.
Click Finish Setup to complete the process.

Step 05

Adding extra storage
If you have lot of users on your server you can quickly run out of storage. That’s why it helps to have additional storage available. OwnCloud supports SMB (Windows share), FTP, WebDAV, OpenStack and Local File system.
To enable extra storage you’ll need to create and edit <ownCloud Root>/config/mount.php. Additional storage can be created either for a single user or a user group.
The following is an example mount.php with all the back-ends enabled:

<ownCloudRoot>/config/mount.php
// Example mount.php showing few of the supported backends enabled
<?php
return array(
// Mount options for group
    ‘group’=>array(
        ‘admin’=>array(
            ‘/$user/files/Admin_Files=>array(
// Accessing Local Filesystem
                ‘class’=>’OC_Filestorage_Local’,‘options’=>array(‘datadir’=>’/mnt/admin_extra_storage’)
            ),
        ),
    ),
//Mount options for users
    ‘user’=>array(
        ‘all’=>array(
// Accessing WebDav Storage
            ‘/$user/files/Web_Dav_Files'=>array(
                ‘class’=>’OC_Filestorage_DAV’,
                ‘options’=>array(
                    ‘host’=>’webdavhost.com/webdav.php’,
                    'user'=>'max',
                    ‘password’=>’secret’,
                    ‘secure’=>true)
                ),
        ),
        ‘user1’=>array(
//Accessing FTP Share
            ‘/user1/files/ftpdownloads’=>array(
                ‘class’=>’OC_Filestorage_FTP’,
                ‘options’=>array (
                     ‘host’=>’ftp.mywebhost.com'
                     'user'=>'max',
                     'password'=>'secret'
                     'root'=>'/ftpfiles')
                 ),
             ),
        )
);

Step 06

Configuring user authentication
If you have lots of users for your cloud, then creating individual users again in ownCloud could become tedious. If you already have a user authentication system in place, ownCloud can use it. OwnCloud supports LDAP, IMAP, SMB, OpenID, WebDAV and FTP. This support is provided in the form of apps. Apps are a way to extend ownCloud’s functionality.
To install additional user authentication back- ends, log into ownCloud, click Settings (gears icon) then Select Apps, then select an App which is not represented with bold fonts, then click Enable in the right pane. Bold means the app is already enabled.
You can use the following apps:
For LDAP: LDAP user and group back-end For OpenID: OpenID user back-end
For WebDAV: WebDAV user back-end
For IMAP, SMB, FTP: user_external
The following shows an example of using the user_external app for authenticating from IMAP, SMB and FTP. You will need to edit the <ownCloudRoot>/config/config.php file.
For IMAP:

‘user_backends’=>array(
  array(
    ‘class’=>’OC_User_IMAP’,
    ‘arguments’=>array(‘{imap.gmail.com:993/imap/ssl}INBOX’)
  )
)

For SMB:

'user_backends'=>array(
  array(
    ‘class’=>’OC_User_SMP’,
    ‘arguments’=>array(‘smbserver’)
  )
)

For FTP:

‘user_backends’=>array(
  array(
    ‘class’=>’OC_User_FTP’,
    ‘arguments’=>array(‘ftpserver’)
  )
)

Step 07

Accessing ownCloud over WebDAV
OwnCloud comes with full WebDAV support. WebDAV is an HTTP protocol that allows read/write file management over the web. The good thing about WebDAV is that the clients are already built into all the popular operating systems, such as Linux, Mac OS X and Windows.
WebDAV is automatically enabled on ownCloud. To access it you will need to use the following URL: www.yoursite.com/ludcloud/ files/webdav.php
Log in with your ownCloud credentials when prompted.
To access your ownCloud account from Nautilus (a popular Linux file manager), you can click File>Connect to Server under Type Select WebDAV and enter server details, click Connect.
You can also use the URL in the format
dav://username@yoursite/ludcloud/files/webdav.php and type it directly into the location bar of Nautilus.
If you want to access it from Mac OS X’s Finder, you can click on Go>Connect to Server… and enter the URL in the format http://www. yoursite.com/ludcloud/files/webdav.php. Enter your credentials when asked.

Step 08

Syncing files using desktop sync clients
If you like Dropbox then you will love this feature. You can use desktop sync clients to sync your files across multiple computers and devices. Desktop sync clients are available for Linux, OS X and Windows. Like ownCloud, desktop sync clients are also open source.
On Ubuntu you can install the package using Apt-get:

$ sudo apt-get install owncloud-client

For other distributions (Debian, CentOS, Fedora, openSUSE etc) you can use the following URL to get the ownCloud desktop sync clients: http://software.opensuse.org/download/pack age?project=isv:ownCloud:devel&package=ow ncloud-client
Download Windows and OS X sync clients from http://owncloud.org/sync-clients/
Desktop sync clients can be used for continuous sync, selective folder sync, multi-folder sync. Multi-folder sync means you can sync multiple ownCloud folders to multiple folder locations. This is an important feature which is not even provided by most popular cloud service providers.

Step 09

Syncing files using mobile sync clients
OwnCloud syncing is not just limited to the desktop. You can use ownCloud’s mobile sync clients to sync your files on the go. OwnCloud mobile apps are available for the Android and iOS (iPhone/iPod touch/iPad) platforms. Both allow you to sync files on the go. The Android version of the app also allows you upload files from any Android app and offers automatic favourite file syncing. The latter feature will keep all your favourite files synced with the mobile device all of the time.
You can either purchase the Android app from the Google Play Store (https://play.google. com/store/apps/details?id=com.owncloud. android) or the Amazon Appstore (www. amazon.com/ownCloud-Inc/dp/B00944PQMK/ ref=sr_1_1?ie=UTF8&qid=1352459188&sr=8- 1&keywords=owncloud). If you are not keen on paying, you can either build the client yourself from the source (https://github.com/owncloud/ android) or download a prebuilt APK file from http://alefzero.eu/a/master.
The iOS version of the App is not open source and is only available in the Apple App Store at https://itunes.apple.com/us/app/owncloud/ id543672169?mt=8. The ownCloud developers say that this is because of Apple’s policy on open source applications in the App Store. The Android and iOS apps cost 63p and 69p (both 99¢) in their respective stores.

Step 10

Extending ownCloud functionality using community add-ons/apps
One of the best features about having your own setup is that you can customise it to your own needs. Using ownCloud, you not only get to customise it but also have the ability to add more features to it. OwnCloud has a vibrant community of people building add-ons for ownCloud, called apps, at apps.ownCloud.com. These apps are available in the Multimedia, PIM, Productivity, Games and Tools categories.
To see how to install apps in ownCloud, in this section we will install the Notes app, which provides notes functionality for ownCloud. Notes is a very capable note-taking app which supports MarkDown and syncing to standalone note-taking apps. You can use Papyrusex, a free note-taking Android app on the Google Play Store (https://play.google.com/store/apps/details?id=com.kallisto.papyrusex) to sync notes. You can also use zNotes (http://sourceforge.net/projects/znotes/) to sync notes on Linux, BSD, OS X and Windows platforms.
Download the Notes zip archive from http:// apps.owncloud.com/content/show.php/ Notes?content=155599. Extract the zip file and copy the extracted folder (in this case, Notes) into the <ownCloudRoot>/apps/ directory. Log into your ownCloud instance as an Admin user, then click Settings>Apps. In the Apps list you will notice a new entry called Notes. Click on it, then click Enable in the right pane to activate the installed app. After activation, refresh the ownCloud page and you’ll see Notes in the left navigation bar. Click on it to start using Notes.
Other apps may require additional steps; check with the documentation of the apps you are installing.

Conclusions

As you can see, creating your own personal cloud has some real benefits in terms of the features and the flexibility. Thanks to ownCloud you can now create the cloud of your dreams using open source software. The ownCloud developers have made sure that you don’t miss your data anywhere by creating sync clients for both desktops and mobile devices. OwnCloud installation is also flexible. As we have seen, you can choose from a wide variety of storage (including other cloud services) as well as plenty of very unconventional authentication mechanisms. If you still think there is something missing in ownCloud that you want, well fear not: you can add that feature yourself. Watch out for a future tutorial on how to write ownCloud apps on Linux User & Developer.