ownCloud Implements Delta Sync Technology

Posted by ownCloud GmbH – 9. April 2018

Launching “Delta Sync,” ownCloud now offers a technology that accelerates the synchronization of updated files. This means that in case of a file change, only the modified parts will be uploaded or downloaded instead of the whole file. Delta Sync significantly reduces the volume of data to be transmitted. The feature was developed by the ownCloud community and will be included in the next major version of ownCloud.

Delta Sync allows ownCloud users to radically reduce the file sizes that have to be uploaded or downloaded from the server after changing a file. Before transferring a file, the software compares its new version with the previous one, and then only replaces the part of the file that has been modified.

For example, if an uncompressed large video file (e.g. 50 GB) had to be synchronized with ownCloud and edited on the server in the past (e.g. by adding a cover image), it would have been necessary to replace the entire 50 GB. Delta Sync now allows you to upload and download essentially only the difference between these two files (the “delta”) – in this case the amount of 10 MB that represents the cover image. The technology is particularly effective for downloads, since only the changes are synchronized here and files are often shared with several users – the potentially saved data volume thus increases parallel to the number of recipients.

Delta Sync is currently in the testing phase and works for uncompressed formats. It is particularly useful for files that have been changed only slightly. The feature is licensed on the server side under AGPLv3 and the client side under GPLv2, and can therefore be freely used and further developed by the community.

Developed by the Community

The demand for the implementation of Delta Sync already existed for several years within the developer community. The interest is accordingly high. On the developer platform, Bountysource, the project was endowed with the highest developer bonus ever offered among all ownCloud projects. Delta Sync was developed by a community member with the support of the ownCloud development team and has been further supported by numerous contributors, some of whom contributed three-figure bounty fees.

The main implementation was the integration of the ZSync algorithm into the ownCloud server and the desktop client. ZSync is mainly used in Linux distributions where rapidly changing beta ISO image files are distributed. It uses the HTTP protocol and .zsync files with precalculated rolling hashes to minimize server load while still enabling diff-transfer transmission for network optimization.

Delta Sync Supports the Following File Formats, and More:

Crypto Container (e.g. VeraCrypt)
Disc images (.iso /.img)
Audio (raw audio formats /.wav /.aiff)
Images (.svg /.tiff /.bmp /.dng)
Video (raw video formats, e.g. uncompressed.avi)
Virtual Machine Images (.vdi /.ova /.img /RAW)
Microsoft PST Files (emails, calendar entries, etc.)
Text Files
Tar Archives

Intensive Testing

The official release of Delta Sync for the ownCloud server and desktop client will take place with the next major release later this year. The testing phase will then begin, in which the tech community will be able to try out the feature and provide feedback. This step is followed by stabilization of the new sync algorithm and further testing, performance measurements and performance optimization, until the feature is finally available to all ownCloud users.

 

 

The diagram shows the synchronization process compared to the previous “full” synchronization.
Source: ownCloud GmbH

Please fill out the form below to get your download.


By submitting this form, you are granting ownCloud GmbH permission to contact you. You can revoke permission any time using the unsubscribe link found at the bottom of every email or by sending an email to info@owncloud.com