Tips for streamlining KDE applications for deployment on Microsoft Windows

In KDE we have a great group of developers hacking on a variety of applications. We usually have no problems getting our software out to end users on Linux because distributions help us packaging and distributing our software.

Where we do have problems, for the major applications, is ensuring our software works on other platforms such as Windows and macOS. This short guide gives a few tips to understand how to make your KDE pet project work on Windows by at least testing it once on said system and verifying a few basic things.

Build your application on Windows

Set up a Windows VM (Windows 7+), a compiler (MSVC2015+ (recommended) -- or let Craft auto-setup MinGW during bootstrap) and build your application using Craft. If set up properly, all you need to do now is to run something along:

craft filelight  

Craft will take care of installing all dependencies required to build filelight (i.e. deps of Qt5, Qt5, deps of KF5, KF5, ...) and only then builds the filelight project.

The end result is one single install root which contains the images of every package you've just built. Now starting filelight is as easy as running this in the terminal:

filelight  

Run the unit tests

Again, easy to do with Craft. If your pet project has some unit tests availabe, you can easily run them by invoking e.g.:

craft --test kcoreaddons  

This will invoke ctest in the correct build directory and run the project's test suite.

Package the application

Craft on Windows has functionality to create installers out of installed packages.

craft --package filelight  

The way this is implemented is pretty simple but powerful

  • Craft collects all files of every image directories of the packages your application depends on
  • Craft collects all files of the application's image directory
  • Craft puts them into an intermediate install root.
  • After that, Craft will strip unneeded files according to blacklists (cf. 'blacklist.txt' and similar functionality in blueprints)
  • After that custom scripts may be run
  • After that makensis is called (from the NSIS installer framework) which basically zips up the whole install root and generates a final installer executable

Things you usually need to fixup

Installer icon

Handled by: Craft

The very first impression counts, so why don't make your installer binary as sexy as possible?

Again let's take the installer generated for filelight. One version without an installer icon set, and one with the filelogo set as logo:

Before: No installer icon
After: With custom installer icon

This is very easy to do with Craft which contains a few helpers to instruct the NSIS installer framework (the scriptable tool we use to generate Windows installers to begin with) properly to our likings.

An exemplary patch in craft-blueprints-kde.git (KDE's blueprint collection for Craft):

commit 1258a4450a1ee2f620856c150678dcaf5b5e7bad  
Author: Kevin Funk <kfunk@kde.org>  
Date:   Mon Nov 20 14:13:04 2017 +0100

    filelight: Add application icon

    Created with:
      convert /usr/share/icons/breeze/apps/48/filelight.svg ./kde/kdeutils/filelight/filelight.ico

diff --git a/kde/kdeutils/filelight/filelight.ico b/kde/kdeutils/filelight/filelight.ico  
new file mode 100644  
index 0000000..1b6a71b  
Binary files /dev/null and b/kde/kdeutils/filelight/filelight.ico differ  
diff --git a/kde/kdeutils/filelight/filelight.py b/kde/kdeutils/filelight/filelight.py  
index 0003f30..ac602c6 100644  
--- a/kde/kdeutils/filelight/filelight.py
+++ b/kde/kdeutils/filelight/filelight.py
@@ -30,6 +30,7 @@ class Package(CMakePackageBase):
         self.defines["productname"] = "Filelight"
         self.defines["website"] = "https://utils.kde.org/projects/filelight/"
         self.defines["executable"] = "bin\\filelight.exe"
+        self.defines["icon"] = os.path.join(self.packageDir(), "filelight.ico")

         self.ignoredPackages.append("binary/mysql")
         self.ignoredPackages.append("libs/qt5/qtdeclarative") # pulled in by solid

This patch adds an ICO file to the repository and references it in the filelight blueprint. Craft takes care of telling NSIS to use this ICO file as the installer icon internally while building your package with craft --package filelight.

Application icon

Handled by: CMake

Imagine starting your application via the Windows Start Menu:

Before: No application icon
After: With custom application icon

For getting the custom application, you need to embrace using Extra CMake Modules ECMAddAppIcon module which provides the CMake function ecm_add_app_icon(...) which in turn allows you to amend your executable with an application icon.

Here's an exemplary patch taken from filelight.git:

commit d7c7f1321547197e5bb9ceba6b8ccc51790bef8b  
Author: Kevin Funk <kfunk@kde.org>  
Date:   Mon Nov 20 23:14:30 2017 +0100

    Add app icon

diff --git a/CMakeLists.txt b/CMakeLists.txt  
index 4c1e5dc..b8b6eda 100644  
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -25,6 +25,7 @@ cmake_minimum_required (VERSION 2.8.12 FATAL_ERROR)
 find_package(ECM 1.3.0 REQUIRED NO_MODULE)
 set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${ECM_MODULE_PATH} ${ECM_KDE_MODULE_DIR})

+include(ECMAddAppIcon)
 include(ECMGenerateHeaders)
 include(ECMInstallIcons)
 include(ECMMarkNonGuiExecutable)
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt  
index 622c9d8..06f5e8d 100644  
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -34,8 +34,16 @@ set(filelight_SRCS
     summaryWidget.cpp
     historyAction.cpp
     mainWindow.cpp
-    main.cpp)
-
+    main.cpp
+)
+set(filelight_ICONS
+    ${CMAKE_CURRENT_SOURCE_DIR}/../misc/16-apps-filelight.png
+    ${CMAKE_CURRENT_SOURCE_DIR}/../misc/32-apps-filelight.png
+    ${CMAKE_CURRENT_SOURCE_DIR}/../misc/48-apps-filelight.png
+    ${CMAKE_CURRENT_SOURCE_DIR}/../misc/64-apps-filelight.png
+)
+ecm_add_app_icon(filelight_SRCS ICONS
+    ${filelight_ICONS})
 ki18n_wrap_ui(filelight_SRCS dialog.ui)
 add_executable(filelight ${filelight_SRCS})

Use Breeze icon theme

Handled by: Craft -- you don't need to do anything.

Breeze-icons (KDE's default icon theme), when configured with te CMake option -DBINARY_ICONS_RESOURCE=ON, installs .rcc files (binary resources, loadable by Qt).

Craft by default passes -DBINARY_ICONS_RESOURCE=ON when the breeze-icons package is installed and thus the RCC file is available by default. When a Craft blueprint using breeze-icons is packaged, the RCC file is automatically included in the resulting artifact.

When the application starts up, KIconTheme scans some directories for RCC files, and if it finds them they're automatically opened and loaded => icons are available.

Further reading about the initial design of the feature by David Faure: https://blogs.kde.org/2016/06/16/icon-theme-deployment-windows-mac-os-and-mobile-platforms

Check presentation of file paths in user interface

Make sure that file paths in your application are rendered consistently. One usual problem we face is that the user interface ends up with strings like C:/Program Files (x86)/KDevelop\, which makes use of forward and backward slashes inconsistently.

Bad: Mixed forward/backward slashes in file paths

Instead, decide for one form of slashes. While on Windows backward slashes are the usual form; the Qt framework makes it a little difficult to print path names with them. API such as QUrl::toDisplayString(...) will always return paths using forward slashes, and one would need to add little helpers everywhere using QDir::toNativeSeparators to do it properly.

At the very minimum use either form; don't mix forward and backward slashes in one file path. Applications on Windows these days handle paths containing forward slashes just fine, by the way.

commit 0de04d386e403ded74554951a8c4dcb9ee9bc1f9  
Author: Kevin Funk <kfunk@kde.org>  
Date:   Mon Nov 20 15:32:54 2017 +0100

    File::fullPath: Nicer file path on Windows

diff --git a/src/fileTree.cpp b/src/fileTree.cpp  
index 9a5d06e..28ed689 100644  
--- a/src/fileTree.cpp
+++ b/src/fileTree.cpp
@@ -21,6 +21,8 @@

 #include "fileTree.h"

+#include <QUrl>
+
 QString
 File::fullPath(const Folder *root /*= 0*/) const
 {
@@ -32,5 +34,6 @@ File::fullPath(const Folder *root /*= 0*/) const
     for (const Folder *d = (Folder*)this; d != root && d; d = d->parent())
         path.prepend(d->name());

-    return path;
+    const QUrl url = QUrl::fromLocalFile(path);
+    return url.toDisplayString(QUrl::PreferLocalFile | QUrl::StripTrailingSlash);
 }

Install C/C++ runtime

Handled by: Craft -- you don't need to do anything.

When installing an application on Windows, the package author also needs to make sure the appropriate C/C++ runtime is injected into the system as part of the installation process. For instance, if your project was compiled using Microsoft Visual C++ 2015 and you want this project to run on another machine, you need to make sure the Microsoft Visual C++ 2015 Redistributable (which contains the C/C++ runtime components) is installed there.

Packages we need on Windows:

  • If project compiled with MSVC:
    • VCRedist installer (contains all necessary libraries)
  • If project compiled with MinGW:
    • Needs another set of libraries (e.g. libstdc++-6.dll, libgccssjlj-1.dll, ...)

But, don't be desperate: Craft has that all covered and will automatically include the binaries for either C++ runtime in the package and make sure it is properly installed as part of the installation of your KDE application on the target machine.

More ideas

If you'd like to know anything else I can probably add a few more paragraphs to this blog post for future reference. Just comment / mail me!

CI for Windows installer and macOS bundle generation: KDE Binary Factory

For some time now the KDE community has had a separate Continuous Integration system running which repeatedly generates Windows installers and macOS app bundles (DMG) for a specific subset of KDE projects.

For a starter, all the KDevelop on Windows releases (read: 32-bit and 64-bit NSIS-based installers) are nowadays generated on this CI and the binary blob which falls out of this process is used as official Windows release installer on the KDevelop website.

So, what exactly does KDE's Binary Factory do, and how does it work -- and why was it created to begin with?

Motivation

With the move to KF5 it became easier to get KDE applications running on non-Linux systems. With the very-much needed split of kdelibs into smaller components, it is now easier to pick & choose what to use on these platforms -- so arguably it is easier to cut out unwanted dependencies (think of DBus, KDE service daemons, ...) on these platforms and stop packaging them at all.

Still, the process of getting a reproducible build of KDE project X on either Microsoft Windows and/or macOS has always been a daunting task. Lots of energy went into KDE's Craft (an open source meta build system and package manager, primarily but not only focused on C++ projects) lately which also helped streamlining this process.

It has never been that easy to generate installers or app bundles for any KDE project (where Craft has a recipe for) using Craft. The missing bit was automating all this by using Craft on Continuous Integration system.

Introducing the KDE Binary Factory

KDE's Binary Factory is a Continuous Integration system primarily created for generating Microsoft Windows installers and macOS app bundles. It has no relation to KDE's CI system other than by sharing some of the machines of the CI workers. The jobs on the Binary Factory are mostly generated manually -- adding projects is very easy though.

How does it work?

First of all, the Binary Factory doesn't really know how to build a particular project. The logic for that -- i.e. which dependencies need to be there, which CMake arguments should be used etc. pp. -- is all stored only in Craft blueprints.

The Binary Factory has a set of projects for which Jenkins will trigger jobs on a nightly basis. It does not do much more than calling Craft like this each night:

# exemplary Windows job for kbruch

# rebuild kbruch
python "C:/Packaging/craftroot32/craft/bin/craft.py"  -v --fetch --unpack --compile --install --qmerge  kbruch

# after that, package kbruch (create a Windows installer)
python "C:/Packaging/craftroot32/craft/bin/craft.py"  -v  --package kbruch  

If there would be missing dependencies for kbruch, or dependencies out of date, Craft would automatically install or update the depedencies, resp.

After the package has been created, Jenkins is instructed to archive the result. You can easily grab the freshly generated installer on the job page, for instance: https://binary-factory.kde.org/job/kbruch-stable-win32/ which usually shows something alike:

Last Successful Artifacts  
    kbruch-17.08.3-windows-msvc2017_32-cl.exe   32.65 MB    view

One click there and the download of the installer starts.

How do I add my pet project?

If you do want your project to be on the Binary Factory, add a recipe (documentation here) and then notify me.

I'd urge you to try building your pet project on Windows yourself first, via Craft, so we don't need to play ping-pong with the CI too much. You (the "project owner") are responsible that the Craft blueprint for your project is up-to-date and works.

Now what to do with the installer/bundle?

The Binary Factory generates Windows installers and macOS bundles at this point. Now what I would not like to see (what already happened) is that individual projects just link to the Binary Factory job pages and tell people 'here, this is the official release of project X'. You shouldn't just link to untested binaries.

Instead I'd like to establish this workflow for project owners:

If there's a release of, say, Kate:

  1. Project owner waits for or triggers a build of Kate on the Binary Factory
  2. Project owner verifies that the installer/bundle works(!)
  3. Project owner uploads the install/bundle on the KDE FTP into the correct project folder
  4. Project owner then adds links to the newly uploaded files on the FTP to the project homepage

Note: Of course linking to the Binary Factory to point users to 'nightly builds of project X' is of course fine -- we do that for KDevelop, too.

Why should I use the Binary Factory?

It has some nice benefits:

  • Automated nightly installer/bundle generation for your project, if set up properly
  • Automated signing of installers/bundles with a KDE-wide code signing certificate
    • Avoids false-positives in AV scanners, warnings by Windows SmartScreen, etc. pp.
  • No need to run a Windows or macOS CI yourself, for package generation
  • Kept up-to-date implicitly via Craft features
    • I.e. right now we're using Qt 5.9.1, KF5 5.37.0
  • It's all there already -- you just need to use it!

Future plans

Start generating macOS bundles

We have a macOS worker set up, but unfortunately the DMG package generation for macOS is still somewhat broken in Craft. We need to sit down and work on this. I don't think there's a lot work left; we can actually create macOS bundles just fine for KDevelop, but there are a few problems with missing shared libraries and/or wrongly set up library metadata (RPATH, etc.).

For the time being, the main focus is on providing Windows installers, but we'll have a look into providing macOS bundles soon-ish now.

Start releasing Windows installers

A couple of KDE projects would be really nice to have on Windows, and actually quite a few installers generated by the Binary Factory are already usable!

One prime example, Filelight:

Setting up a job on the Binary Factory for generating the installer for Filelight literally took me a few hours (for preparing the Craft blueprints, injecting the job in Jenkins -- that's it). Now we just need to link the installer on the official Filelight homepage.

Final words

I'm happy to see this finally working in an automated way. At least for the KDevelop team, the automated installer generation for the KDevelop installers has been a major step into the right direction. After a couple attempts of doing that on personal machines where we literally always managed to break our setup, we now have a clean, automated process of generating them on a isolated machine.

I hope the Binary Factory can be useful to other KDE projects as well. I'm happy to help you guys out to set up jobs on the CI so your personal projects can be built.

KDevelop for Windows available on Chocolatey now

You might have noticed there's KDevelop for Windows out now...

Which is already great in itself! But now it's also possible to install it via the super popular Windows package manager for Windows, Chocolatey.

Here's all you need (in case you already have Chocolatey installed on your system):

  • Start a Command Prompt as Administrator user
  • Type in choco install kdevelop to install
  • Start KDevelop via the Windows start menu

That's it!

Note: KDevelop does not ship a C++ compiler, you'll need to either setup MinGW/GCC, Microsoft VC++ or even Clang on Windows. Setup instructions here: https://userbase.kde.org/KDevelop4/Manual/WindowsSetup

Here's what choco it does:

C:\WINDOWS\system32>choco install kdevelop  
Installing the following packages:  
kdevelop  
By installing you accept licenses for the packages.

kdevelop v5.0.2 [Approved]  
Downloading kdevelop 64 bit  
  from 'http://download.kde.org/stable/kdevelop/5.0.2/bin/windows/kdevelop-5.0.2-x64-setup.exe'
Progress: 100% - Saving 90.45 MB of 90.49 MB (94839304/94887333)  
Download of kdevelop-5.0.2-x64-setup.exe (90.49 MB) completed.  
Hashes match.  
Installing kdevelop...  
kdevelop has been installed.  
Added C:\ProgramData\chocolatey\bin\kdevelop.exe shim pointed to 'c:\program files\kdevelop\bin\kdevelop.exe'.  
 The install of kdevelop was successful.
  Software installed as 'EXE', install location is likely default.

Chocolatey installed 1/1 packages. 0 packages failed.  
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Check out Pro / Business for more features! https://chocolatey.org/compare

C:\WINDOWS\system32>where kdevelop  
C:\ProgramData\chocolatey\bin\kdevelop.exe  

Happy hacking!

Thanks a lot to Hannah von Reth for setting up the KDE Chocolatey category, including publishing the KDevelop package there!

As always, we ask you to provide us with any kind of feedback on the Windows version! It's much appreciated.