Archive for July, 2016

Transfer Public Links to Federated Shares


Transform Public Links to Federated Shares

Transform a public link to a federated share

Creating public links and sending them to your friends is a widely used feature of Nextcloud. If the recipient of a public link also has a Nextcloud or ownCloud account he can use the “Add to your Nextcloud” button to mount the content over WebDAV to his server. On a technical level all mounted public links use the same token, the one of the public link, to reference the shared file. This means that as soon as the owner removes the public link all mounts will disappear as well. Additionally, the permissions for public links are limited compared to normal shares, public links can only be shared read-only or read-write. This was the first generation of federated sharing which we introduced back in 2014.

A year later we introduced the possibility to create federated shares directly from the share dialog. This way the owner can control all federated shares individually and use the same permission set as for internal shares. Both from a user perspective and from a technical point of view this lead to two different ways to create and to handle federated shares. With Nextcloud 10 we finally bring them together.

Improvements for the owner

Public Link Converted to a Federated Share

Public link converted to a federated share for bjoern@myNextcloud.net

From Nextcloud 10 on every mounted link share will be converted to a federated share, as long as the recipient also runs Nextcloud 10 or newer. This means that the owner of the file will see all the users who mounted his public link. He can remove the share for individual users or adjust the permissions. For each share the whole set of permissions can be used like “edit”, “re-share” and in case of folder additionally “create” and “delete”. If the owner removes the original public link or if it expires all federated shares, created by the public link will still continue to work. For older installations of Nextcloud and for all ownCloud versions the server will fall-back to the old behavior.

Improvements for the user who mounts a public link

After opening a public link the user can convert a public link to a federated share by adding his Federated Cloud ID or his Nextcloud URL

After opening a public link the user can convert it to a federated share by adding his Federated Cloud ID or his Nextcloud URL

Users who receive a public link and want to mount it to their own Nextcloud have two options. They can use this feature as before and enter the URL to their Nextcloud to the “Add to your Nextcloud” field. In this case they will be re-directed to their Nextcloud, have to login and confirm the mount request. The owners Nextcloud will then send the user a federated share which he has to accept. It can happen that the user needs to refresh his browser window to see the notification.
Additionally there is a new and faster way to add a public link to your Nextcloud. Instead of entering the URL to the “Add to your Nextcloud” field you can directly enter your federated cloud ID. This way the owners Nextcloud will send the federated share directly to you and redirect you to your server. You will see a notification about the new incoming share and can accept it. Now the user also benefit from the new possibilities of the owner. The owner can give him more fine grained permissions and from the users point of view even more important, he will not lose his mount if the public link gets removed or expires.

Nextcloud 10 introduces another improvement in the federation area: If you re-share a federated share to a third server, a direct connection between the first and the third server will be created now so that the owner of the files can see and control the share. This also improves performance and the potential error rate significantly, avoiding having to go through multiple servers in between.

History and Future of Cloud Federation


Federated Cloud Sharing - Connect self-hosted, decentralized clouds

Federated Cloud Sharing – Connect self-hosted, decentralized clouds

I’m now working for about two years on something called Federated Cloud Sharing. It started on June, 23er 2014 with the release of ownCloud 7.0. Back then it was simply called “Server to Server sharing”. During all this years I never wrote about the broader ideas behind this technology, why we do it, what we achieved and where we are going.

Motivation

The Internet started as a decentralized network, meant to be resilient to disruptions, both due to accidents or malicious activity. This was one of the key factors which made the Internet successful. From the World Wide Web, over IRC, news groups, e-mail to XMPP. Everything was designed as decentralized networks, which is why if you are on the Google servers you can email people at Yahoo. Everybody can set up his own web server, e-mail or chat server and communicate with everyone else. Individuals up to large organisations could easily join the network, participate and build business without barriers. People could experiment with new innovative ideas and nobody had the power to stop them or to slow them down. This was only possible because all underlying technology and protocols were build on both Open Standards and Free Software.

This changed dramatically over the last ten years. Open and inclusive networks were replaced by large centralized services operated by large companies. In order to present yourself or your business in the public it was no longer enough to have your own website, you had to have a page on one or two key platforms. For communication it was no longer enough to have a e-mail address, or be on one of the many IRC or XMPP servers. Instead people expected that you have a account on one of the major communication platforms. This created huge centralized networks, with many problems for privacy, security and self-determination. To talk to everybody, you have to have an account on Facebook, at Google, Skype, Whatsapp, Signal and so on. The centralization also made it quite easy to censor people or manipulate their view by determining the content presented to them. The algorithms behind the Facebook news feed or the “what you missed” in Twitter are very clever — or so we assume, as we don’t know how they work or determine what is important.

The last few years many initiatives started to solve this problem in various ways, for example by developing distributed social networks. I work in the area of liberating people who share and sync all sort of data. We saw the rise of successfully projects such as ownCloud, Pydio and now of course Nextcloud. They all have in common that they built Free Software platforms based to a large extend on Open Standards to allow people to host, edit and share their data without giving up control and privacy. This was a huge step in creating more competition and restoring decentralized structures. But it also had one big drawback. It created many small islands. You could only collaborate with people on the same server, but not with others who run their own server. This leads us to the concept of federated cloud sharing.

Server to Server Sharing

The first version of this ideas was implemented in ownCloud 7.0 as “Server to Server Sharing”. ownCloud already knew the concept of sharing anonymous links with people outside of the server. And, as ownCloud offered both a WebDAV interface and could mount external WebDAV shares, it was possible to manually hook a ownCloud into another ownCloud server. Therefore the first obvious step was to add a “Add to your ownCloud” button to this link shares, allowing people to connect such public links with their cloud by mounting it as a external WebDAV resource.

Federated Cloud Sharing

Server to server sharing already helped a lot to establish some bridges between many small islands created by the ability to self-host your cloud solution. But it was still not the kind of integration people where used to from the large centralized services and it only worked for ownCloud, not across various open source file sync and share solutions.

federated-cloud-id

The next iteration of this concept introduced what we called a “federated cloud ID”, which looks similar to a e-mail address and, like email, refers to a user on a specific server. This ID could then be used in the normal share dialog to share files with people on a different server!

share dialog - federated cloud id

The way servers communicate with each other in order to share a file with a user on a different server was publicly documented with the goal to create a standardized protocol. To further the protocol and to invite others to implement it we started the Open Cloud Mesh project together with GÉANT, an European research collaboration initiative. Today the protocol is already implemented by ownCloud, Pydio and now Nextcloud. This enables people to seamlessly share and collaborate, no matter if everyone is on the same server or if people run their own cloud server based on one of the three supporting servers.

Trusted Servers

In order to make it easier to find people on other servers we introduced the concept of “trusted servers” as one of our last steps. This allows administrator to define other servers they trust. If two servers trust each other they will sync their user lists. This way the share dialogue can auto-complete not only local users but also users on other trusted servers. The administrator can decide to define the lists of trusted servers manually or allow the server to auto add every other server to which at least one federated share was successfully created. This way it is possible to let your cloud server learn about more and more other servers over time, connect with them and increase the network of trusted servers.

federation

Open Challenges: where we’re taking Federated Cloud Sharing

Of course there are still many areas to improve. For example the way you can discover users on different server to share with them, for which we’re working on a global, shared address book solution. Another point is that at the moment this is limited to sharing files. A logical next step would be to extend this to many other areas like address books, calendars and to real-time text, voice and video communication and we are, of course, planning for that. I will write about this in greater detail in on of my next blogs but if you’re interested in getting involved, you are invited to check out what we’re up to on GitHub and of course, you can contact me any time.