ownCloud development over December

New in ownCloud: favorites

As the second half of December took many people on holiday the bi-weekly reports have been folded into a full December overview. Despite the holiday period, with over 1100 closed issues including no less than 395 merged pull requests, this was not a quiet time. Certainly this has to do with the feature freeze which went in effect on January 1st! All features not in at that date will have to wait until ownCloud 8.1, while our focus is now on stabilizing and bug fixing. As usual, this report pulls together activities from development mailing lists, blogs, coding work and everything else community members share with us. Tips are very much welcome! Ping Jos with your input.

To catch up to what happened before, read the development update over the second half of November.

Highlights in December include the huge amount of work in Core merging features like favorite support in the Files app, improved server-to-server sharing, app-specific search, improved user management and more. There was also a a lot of accessibility work to make using ownCloud with the keyboard easier and groundwork was being laid for some serious changes to file handling by the desktop client and WebDAV. In app news, many improvements entered the Android app including the ability to sync entire folders. And of course the feature freeze went into effect on January 1st.

Core

Core had almost 200 PR’s merged, with a series of big new features making it in before the feature freeze. This includes favorite support in the Files app, improved server-to-server sharing, app-specific search, enhanced user management, UI design improvements and improved accessibility and many enhancements to the ownCloud API for app developers.

New features

The following feature improvements were merged:

• This PR adds the ability to favorite files and other data in ownCloud! A lot of work was done to prepare this, with this which made it easier to add the favorites logo in the user interface, this PR made it possible to request tags over WebDAV and this added the ability to search by tags to view, storage and cache in ownCloud. Also, a new app was created to handle tags and favorites in ownCloud. This PR added favorites to the files app and finally our design team went over the feature and polished it up. More work in this area is coming for ownCloud 8.1
• As parts of efforts to improve the control of administrators over external storage, a public api was introduced which apps can use to add additional mount points in a more structured way then listening to hooks. Mount specific options were also added to the mount configuration, including a storage id, to make it easier to improve the configuration UI. Much more improvements are still underway to future releases
• As part of the efforts of the new generation sever-to-server sharing an OCS API for sharing was implemented. It prompted a debate on the use of the OCS API in the first place, something that got a follow-up on the mailing list and then moving to github. The UI for NG server2server sharing was implemented in this PR followed by changes in the calendar and contacts app. To make activity show up nicely on in the Activity app, some core changes were made and a PR went in for the Activity app
• On a slightly related note, you’ll now get a notification in the Activity app when a publicly shared file or folder is downloaded
• Following plans from issue 821 about improving the design and implementation of search in ownCloud, butonic made search application specific. This makes a search in for example the Files app only show files, while a search in the Contacts app shows contacts! He implemented this change also in the calendar, contacts and bookmarks apps. The Files app now also shows results in the main content area instead of just in the search pop-up and improvements to query parsing. More improvements are coming in a future release
• LDAP search was improved with the ability to use the occ command line tool to look for LDAP users and groups.
• A big LDAP user cleanup was merged in both ownCloud 7 and 8, enabling the deletion of users and making LDAP work more reliable and a very small change by lyonel makes the ldap user plugin a bit less noisy
• A performance improvement for LDAP went in: caching total user count saves a lot of queries when loading the user interface
• The big plan for overhauling user management has been partially implemented, with more coming for ownCloud 8.1. The following features did make it in 8.0:
  • This pr added a REST route for managing groups and users. Another item related to the underlying code managing users and groups which got merged is this, and here is another one
  • The user back-end is now shown in user management, possible because user backends now have names
  • Another smaller improvement is that now even on big screens with room for many users, the user list is populated properly
  • The mail address of users can now be changed
  • Sys administrators will love the new feature to send an email to new users with their name and the ownCloud URL  (the template editor can deal with these mails, so they are easy to configure)

user management showing user back end

For developers

For developers, a series of changes will have a big impact on how they develop apps:

• Users and developers should know that PHP 5.4 is required for ownCloud 8.0! This quickly resulted in the removal of the first 5.3 workaround and more with this and this.
• In the a-really-big-deal-for-app-developers-department, this PR adds dependency management for apps for php extensions, command line tools, operating systems and owncloud version
• This ‘Intelligent container’ can figure out dependencies by itself
• The improved FileActions JS to allow for custom rendering (also needed for the ability to favorite files and other content)
• ownCloud extended the tags API by adding getTagsForObjects in ITags
• This PR creates a logger that uses PHPs error_log to handle logging, making it easier to deal with logs for app developers
• This PR added the ControllerMethodReflector to public API (by oparoz)
• ownCloud added a missing public interface for iOS client app id
• After a bit of a debate to find out if this was the right approach, a simple plugin system for Javascript was added, making it possible for apps to inject code into certain classes.
• Old and replaced app framework API’s got deprecated
• Raydiation added an interface for the timefactory class and an interface for the root folder so apps have a way to deal with storage
• A big configuration cleanup took place: the OC_Preferences refactoring touched files all over the ownCloud code
• Always on the lookout for things to remove, Lukas found an unused file
• The timezone logic was moved out of server.php to keep this base file simpler and thus less likely to break
• The ownCloud development tool, a Python 3 tool to build new and improve existing ownCloud apps, had several fixes and improvements like a move to the new array syntax in PHP 5.4 for ownCloud 8 and a host of other improvements. If you want to build an ownCloud app, be sure to check out this tool!

Design improvements

ownCloud aims to present a consistent, easy to use and pleasant experience to users, and the design team plays a big role in making that possible. In December, the following changes were merged in ownCloud:

• The border was removed from the user menu, adjusting it to to apps menu
• The bitmap and vector mime icons were improved by oparoz
• szlwzl added @ sign between email address and domain namein the admin settings
• Handling of read only shared files was improved, giving a clear indication in the UI what is going on
• The update error page might not be liked much, but at least looks a bit more consistent now
• To improve the accessibility of ownCloud, keyboard navigation was reworked
• A lot more accessibility improvements were made in this PR and more is coming
• This PR improves the ’empty state’ of pretty much all screens, like favorites, all files, shared with you, shared by you and so on.
• The wording in the shared dialog got fixed
• To fit better with the rest of ownCloud, Documents now uses a sans-serif font by default

Other changes

Some other changes include the exciting work on ‘direct download links’ which would allow the ownCloud client and other external tools to bypass ownCloud, getting data directly from external storage, which is very much a Work In Progress but can bring large performance improvements; and the similar work on direct file handling over WebDAV which brings chunked syncing (“only syncing file changes”) a little closer.

• To improve performance of the clients with a wide variety of external storages, ownCloud gained the ability to give a direct download link to the client instead of piping all data through ownCloud. See more details here. For now only the base of this feature is here, much of the hard work still has to be done as there are many protocols to be supported thanks to the external storage support in ownCloud, and S3 and SWIFT are the first in the list of priorities (largely due to the fact they are HTTP based and, unlike FTP and most other protocols, usually not firewalled). This will be worked on in the ownCloud 8.1 timeframe
• When handling files over WebDAV, one can now also work directly with storage, saving significant time and resources in ownCloud. But this change also requires significant changes, especially to the encryption code so this is still a WIP and a more complete implementation will come with ownCloud 8.1. Note that this is also a first step towards chunked upload – something clearly many users want it as there’s a bounty of over USD 1000 on this already. As always – if you are interested in hacking on this, join the conversation! These and similar changes planned/under discussion related to storage are being tracked here
• credentials are now only stored in the session if required
• This PR reorganized the encryption folder structure to make encryption more robust. This simplifies the code, speeds up operations but touches encryption all over so it can use some good testing!
• The passwordsalt config value in ownCloud was deprecated – developers are urged to use the ICrypto interface.
• AdamWill  made Google Drive external storage more robust by improving the dealing with non-unique filenames and folders. GDrive supports having several files and folders with the same name – but this confuses ownCloud.
• improvements in the authentication part of logging fixes some known and perhaps even unknown issues
• Performance of reading encrypted files got faster thanks to caching the keys

New and improved ownCloud apps

As usual, the ownCloud apps, both those included in ownCloud or otherwise hosted in our github repo as well as those on apps.owncloud.com were the subject of much attention and improvements. We should point out that many app developers push code directly to their apps rather than working via Pull Requests and thus their work goes unmonitored by this development digest. There is a lot more work going in than you read here!

If you, as developer, would like to ensure users and other developers can follow what is going on, send me an occasional email with what you’ve been up to!

• We renamed the ‘mirall’ repository to ‘client’
• The Notes app works better on mobile devices
• The Calendar app got a number of small design improvements
• jakobsack moved the creation of default address book to the back end in Contacts
• LEDfan merged a fix for Contacts with shared address books
• New is “show preferred value in contacts list” by ymv and by fixing caching in the contacts app, slowness in the Chat app should be solved
• Talking of which, the Chat app now shows user names if no icons are set
• As ownCloud can’t really deal with TIFF files at the moment, this PR makes it ignore these files in the Gallery app, which also got some accessibility improvements and a fix of the progress bar
• The Music app received two useful improvements, one allowing users to drop metadata so a rescan can take place and another to allow users to manually start the update of the music collection. The latter patch also made sure the initial scan doesn’t start without first asking the user.
• In the Mail app, zinks- fixed an issue showing folders by restructuring the way they are shown
• arkascha prepared a release for Shorty version 0.4.5 and Shorty Tracking 0.2.4 – later on, 0.5.0 and 0.3.0 saw the light of day and it is high time to get these new releases!
• kwillems merged in a number of improvements to the build scripts for the desktop client he’s been adding and improving things for a while now – great work!
• Following a number of mockups by YokoOh and with feedback from other ownCloud designers, the header bar in the Documents app was improved by VicDeo
• Unfortunately a bigger update of WebODF, which would bring many improvements to Documents, did not make it in
• In preparation for multiple accounts, danimo refactored and improved the settings dialog of the desktop client
• Based on an idea (and some code) from jdsantiagojr, ganomi improved the look of the Bookmarks app to be more like the File app. ganomi also fixed Amazon bookmarks while TtuxX removed the 10 tag limitation from bookmarks
• The News app keeps moving forward with fixes for a lot of articles (from keinkurt) and other enhancements
• The PDFviewer was updated to work with ownCloud 8 and got some other fixes

The Android Client

I highlight the Android client because it merged many improvements, the most important of which are below.

• The Android app now shows a file count thanks to the work by tobiasKaminsky. He’s been busy, and also got these changes merged:
  • a change so the account list is a radiobutton list
  • enabling the ‘open with’ context menu on downloaded files
  • rotate image changes merged
  • this media scanning fix
• Some fixes and improvements from LukeOwncloud went in here, improving the handling of logs
• ekeitho contributed a change to put a subject in mails used to share files with the Android client
• Kernald fixed the displaying of an image upon unlocking the phone which got merged here
• It is now possible for the app to synchronize entire folders (groundwork here and here) this downloading now happens in a separate worker thread to keep the app responsive and added a cancel option to folder download, added to the menu here  and as icon on folders

In the app store

On apps.owncloud.com:

• New is the files_hubic app which provides external storage support for the hubic file storage service
• Also new is OPDS Catalog, bringing OPDS support to ownCloud so users can access books and other electronic publications from their ownCloud on OPDS compattible devices like FBReader/CoolReader/Aldiko on Android or Marvin/KyBook/Stanza on iOS. The OPDS Catalog author quickly released a series of updates so the app is (at the time of this writing) at 0.4
• And third newcomer is the Pre-Login Validator. This tool allows you to allow or deny users at login. After a proof-of-concept on December 1st, an update was made with a series of rules later in the month. Check it out!
• The user_servervars app was updated with a avatar mapping, plugin table and removal of group binding references. This App provides with binding between HTTP server variables and user attributes required for some SSO platforms.
• The Ampache server was updated to 0.4.4
• ownCloud SMS got some minor fixes
• The News app updated to 4.3.2 as a preview for ownCloud 8 (but can be installed on ownCloud 7 – note that it requires some manual work).
• Passman also updated and is now at 1.3, bringing a variety of bugfixes

If you want to get involved in coding on ownCloud, be it on apps or in some other way – check out this page!

Mailing lists and planet ownCloud

Some of the many changes for developers as mentioned in the Core section above were also discussed on the mailing list: deprecated functionality, and another one on that subject, work on improving php classes and tests, changed paths of vendor JS libraries, and more. Other notable conversations include:

• A notification came that the documentation is open for ownCloud 8 work
• A notification that the Mirall repo was renamed to client
• There was a conversation about trusted_domains and home networks
• Pierre wanted to create a new external storage and asked how to do so
• Releases: ownCloud client 1.7.1RC1, owncloud client 1.7.1 Beta, ownCloud 7.0.4 RC1 and 7.0.4 Final

You can join the discussions on this page.

On December 5th, we had a bug triaging day, which was announced on the testpilots list. Otherwise, discussions on the testpilots list centered around testing releases and results.

The events mailing list hosted conversations about Chemnitzer Linux Tage in Germany, a GSOC meetup in Stuttgart and an ownCloud assembly at CCC. If you think events are missing and you want to meet fellow ownCloud users and developers at ownCloud meetups or see us at conferences, find a calendar of ownCloud events on owncloud.org/events. check out this page on how to get involved.

On Planet ownCloud:

• Klaas Freitag blogged about ownCloud Dolphin integration
• ownCloud 7.0.4 was announced
• We welcomed Google to the world of ODF support, wrote a blog about extending ownCloud with apps and shared the development update on the 2nd half of November
• ownCloud Inc. blogged about the FCC and Data protection policy,  three ways to block Dropbox in the workplace and a Wired article
• And just before the year was over, Frank shared his reflections on 2014 and 2015 for ownCloud and data freedom in general

Please note that if you’re an ownCloud contributor, you should get your blog aggregated on owncloud.org/news! Ping Jos with your RSS feed.

We hope you enjoyed the read!