What's new in KDevelop 5.0?

August 23, 2016 | Tags: kdevelop, kde, kdevelop-release

Almost two years after the release of KDevelop 4.7, we are happy to announce the immediate availability of KDevelop 5.0!

Screenshot showing KDevelop 5.0 under Linux

While the release announcement on kdevelop.org is kept short intentionally, this blog post is going more into depth, showing what's new in KDevelop 5.0.

Read on...

Changes in language support

C++ support powered by Clang

We replaced our legacy C++ parser and semantic analysis plugin with a much more powerful one that is based on Clang from the LLVM project.

A little bit of history: KDevelop always prided itself for its state of the art C++ language support. We introduced innovative code browsing functionality, semantic highlighting and advanced code completion, features that our user base has come to rely upon for their daily work. All of this was made possible by a custom C++ parser, and an extremely complex semantic analyzer for the C++ language. Adding support for all the quirky corner cases in C++, as well as maintaining compatibility with the latest C++ language standards such as C++11, drained our energy and stole time needed to improve other areas of our IDE. Furthermore, the code was so brittle, that it was close to impossible to improve its performance or add bigger new features such as proper C language support.

Now, after close to two years of work, we finally have a solution to this dilemma: A Clang based language plugin. Not only does this give us support for the the very latest C++ language standard, it also enables true C and Objective-C language support. Furthermore, you get all of the immensely useful compiler warnings directly inside your editor. Even better, fixing these warnings is now often just a matter of pressing a single button to apply a Clang provided fix-it!

Screenshot of KDevelop showing Clang fixits

There are, however, a few caveats that need to be mentioned:

On older machines the performance may be worse than with our legacy C++ support. But the new Clang based plugin finally scales properly with the number of cores on your CPU, which can lead to significantly improved performance on more modern machines.
Some features of our legacy C++ support have not yet been ported to the new plugin. This includes special support for Qt code, most notably signal/slot code completion using the old Qt 4 macro syntax. We will be working on improving this situation and welcome feedback from our users on what we should focus on.
The plugin works fine with Clang 3.6 and above, but some features, such as auto-type resolution in the code browser, or argument code completion hints within a function call, require newer versionsof Clang. The required changes have been contributed upstream by members of our community and we intend on continuing this effort.

Another screenshot to make you want to try KDevelop 5.0 instantly:

Screenshot of KDevelop analyzing doxygen-style code comments (KDevelop analyzing doxygen-style code comments)

For the best C++ experience in KDevelop, we recommend at least Clang 3.8.

CMake support

We removed the hand-written CMake interpreter and now leverage meta data provided by upstream CMake itself. The technology we're building upon is a so called JSON compilation database (read more about it in this insightful blog post). Technically, all you need to do is to run cmake with the -DCMAKE_EXPORT_COMPILE_COMMANDS flag, and CMake will take it from there, emitting a compile_commands.json file into your build directory.

KDevelop now supports reading those files, which is way more reliable than parsing CMake code ourselves.

But this step also means that we had to remove some of the useful advanced CMake integration features, such as the wizards to add files to a target. We are aware of this situation, and plan to work together with upstream to bring back the removed functionality in the future. Hopefully, you agree that correctness and much improved performance, where opening even large CMake projects is now close to instant, makes up for the loss of functionality.

QML/JavaScript support

With KDevelop 5, we decided to officially include support for QML and JavaScript code. This functionality has been worked on for years in our playground and now, we finally incorporated these experimental plugins and will start to officially support them.

Screenshot showing KDevelop's QML support

Our thanks go to the Qt Creator community here, as we leverage their QML and JavaScript parser (QmlJS, see here) for our language support plugin.

Screenshot showing KDevelop's QML support

QMake support

With KDevelop 5, we decided to officially include support QMake projects, too. Same story here, this functionality has been worked on for years and we now start to officially support them.

The new KDevelop plugin for QMake is simplistic but already super useful for many projects. If you are using more complicated QMake features and want to improve our interpreter, please get in touch with us!

Python, PHP, ...

Together with all this, KDevelop 5 will continue to officially support Python 3 and PHP. In our playground we also have support for Ruby, and there are plans to integrate basic Go and Rust support. If you are interested in improving support for any of these languages in KDevelop, please step up and start contributing!

Screenshot of KDevelop's Python support

Other changes

Remove assistant overlay in favor of navigation widget

Another major thing we worked on was rethinking KDevelop’s assistant popup; especially in the current 5.0 betas, it tended to be a bit annoying and got in the way a lot. We thus removed the assistant overlay in favor of offering executions of assistants from the navigation widget.

Here's a screenshot of the assistants in form of a navigation widget:

Screenshot of KDevelop's new assistant widget

Key changes:

No longer automatically popup a widget whenever there's a problem (distracting!)
Only popup when invoked (via Alt, or via mouse hover)
Show problems on keyboard activation (via Alt, wasn't possible before)
We can use more text in the solution assistant descriptions (since we requested them, we can cover more space implicitly)
No longer creates a OpenGL context each time there's an error (this has been slow at times, using the old assistant popup. There was a noticable lag while typing on heavy load)

Per-project widget coloring

Thanks to Sebastien Speierer we got a super useful feature into KDevelop 5.0: Widget coloring based on an items affinity to a project.

A picture is worth more than a thousand words, see it in action here:

Screenshot showing KDevelop's per project widget coloring

As you can see, both the project explorer rows as well as the document tab bar items are colored based on the project affinity. This is useful to quickly decide which project a specific file belongs to.

(Note this feature is optional, it's possible to enable/disable in settings)

Progress reporting of make/ninja jobs

We added support for tracking the progress of make/ninja jobs in KDevelop, we do so by simply parsing the first few chars of the output of make and ninja. For make, this will only work for Makefiles generated by CMake so far, as those contain proper progress information). Thus, this feature won't work when make is invoked on Makefiles generated by QMake.

Screenshot:

Screenshot showing KDevelop reporting ninja's progress

The progress bar on the bottom right indicates the progress of the ninja invokation. Extra gimmick: Starting with Plasma 5.6, this progress is also indicated in the task bar entry of your task switcher in the Plasma shell.

Welcome Page redesign

The welcome page (the widget which is shown whenever you have no tabs open in KDevelop) got redesigned to better match the current widget style in use). Screenshot:

Screenshot of KDevelop's welcome page plugin

Various debugger related improvements

Debugger support is KDevelop's unloved child, but it got some improvements in 5.0, and will get quite a few improvements in the upcoming 5.1 release (due to the LLDB GSoC happening, which also touches lots of debugger agnostic code).

Debugger support in 5.0 was improved by simply streamlining the debugger related widgets where possible.

Screenshot of KDevelop's frame stack tool view

Changes:

Frame stack model: Non-existing files are now rendered in gray
Frame stack model: Pretty urls for file paths (i.e. myproject:src/main.cpp), elided in the middle now
The crashed thread is now highlighted properly
A lot more

Splash screen removal

For performance reasons the splash screen got removed in 5.0. There's been a short discussion on the KDevelop development mailing list about the pros and cons, in the end we decided to drop it.

The reasons for dropping it were:

Perfomance: Our QML-based splash screen actually had a noticeable impact on the start time of KDevelop (kind of defeated its purpose)
Feels old-fashioned: Showing a splash screen always makes me feel a bit nostalgic, it's just not a modern way to indicate that your application is starting up. All modern DEs provide a way to indicate this (i.e. by a bouncy cursor in Plasma, good old hourglass in Windows -- and OS X has animations for this as well).
Startup time got improved significantly (see more about that below) during 4.x -> 5.x, so it no longer felt necessary

Under the hood

Just an excerpt:

We have ported our huge code base to Qt 5 and KDE frameworks 5 (KF5).
We cleaned up many areas of our code base and improved the performance of some work flows significantly.
(Cold) start performance of KDevelop got improved significantly due to changes in KDevelop and libraries below (KF5 icon loading, KF5 plugin loading, etc.) -- something in the order of several seconds on my test machine (Lenovo T450s).

Just to get you an idea how much work was put into the 5.0 release over the years:

kdevplatform% git diff --stat origin/1.7 v5.0.0 | tail -n1  
 1928 files changed, 65668 insertions(+), 73882 deletions(-)

kdevelop% git diff --stat origin/4.7 v5.0.0 | tail -n1  
 1573 files changed, 131850 insertions(+), 30347 deletions(-)

Get it

Linux AppImage

If you're on Linux you can start using KDevelop right away, by downloading & running the new KDevelop 5.0 AppImage.

Other platforms

With KF5 overall cross-platform support of KDE applications got better by order of magnitudes. Tons of hours have been spent improving OS X and Windows support.

We hope to release an official OS X app bundle & a Windows installer package soon.

Read more about other installation instructions.

Verdict

We're super proud to finally release KDevelop 5.0 to the public! We think it's a solid foundation for future releases.

With the use of Clang as the C++ support backend we hope to be able to put more energy into the IDE itself as well as other plugins instead of playing catchup with the C++ standard!

Happy to hear your opinions about KDevelop 5.0. What do you like/dislike?

KDE on Windows status update

June 18, 2016 | Tags: kde, kde-windows, emerge, kderanda16

Disclaimer: No, it's not dead :)

During Randa we've discussed the KDE on Windows road map. There's been a brisk involvement of the Randa Meetings participants in the platform discussion sessions on Tuesday.

Here's a brief summary of what's been going on.

Installer creation

Traditionally the KDE on Windows project has been focused on providing one single installler which provides the whole KDE experience in one installer (that means, having one installer which was capable of installing all of KDE, even including the Plasma shell). We've (finally?) come to conclusion this is not what the average Windows users wants. People tend to install the application they want, but nothing else.

Our plan now is to provide single application installers for KDE software. That means having an installer which just installs Kate, or Krita, or KDevelop, or Marble, whatever. No "KDE installer" where you can select the individual applications.

We're just at the beginning of the initiative (with lots of bug fixing going on behind the scenes), but here's the current list of applications being available on Windows:

Hannah von Reth is working hard on creating even more installer packages!

Continuous Integration on Windows

We're working on Windows CI for KDE. More precisely, that means we're turning the original KDE CI infrastructure into a system where we can also build the KDE projects on Windows. This is all work in progress, but we expect to have the first set of jobs running on our sandbox CI system running until the end of the week.

Up to now, our plan is to reuse the existing Emerge infrastructure for CI as much as possible, which also covers building all the dependencies of Qt5.

Emerge

Emerge is a tool to build the KDE sources and its third-party requirements on MS Windows. Traditionally. It's way more than that today: With Emerge it's possible to build any project if you just provide a recipe for it, highly similar to comparable tools such as Homebrew on OS X. If you have a complex dependency chain, and need a user space package manager (which builds from source), then Emerge is the tool to use.

During Randa we have polished the OS X and Linux ports of emerge, making it super easy to build projects including all its dependencies even on those platforms. The support is not official yet, it's still work in progress. We'll officially announce as soon as this gets ready. Just one example: we've successfully built Kate on OS X using Emerge with a stock Qt5.

What else has changed in Emerge:

Almost all custom Qt patches could be dropped in emerge.git
- Most of the missing functionality is now upstream
Qt 5.6.1 used by default in the Qt recipes
Breeze-icons recipe now installs icons as RCC (see David Faure's blog for more information about this feature)
Emerge prepared for being integrated as build tool in the Windows CI
- Now possible to install dependencies, store build artifacts in zip files, etc.
- Build artifacts can be re-used for other builds

Landing page

We've also cleaned up the KDE on Windows landing page a lot:
https://community.kde.org/Windows

Tons of information the average user isn't interested in got removed or moved to sub pages.

Verdict

KDE on Windows is not dead, as some people from the outside tend to think. It's true we're no longer willing to maintain a "KDE installer" as such, but instead focus on bringing individual apps which are ready on Windows to this platform.

Keep your eye on https://community.kde.org/Windows to get notified about new packages.

Actually, a big thank goes to Hannah von Reth, who's the actual maintainer of Emerge, who does a great job bringing this super useful tool forward and does a great job fixing tons of Windows issues in KDE projects.

Apart from that: Which favorite KDE application would you like to install yourself on Windows?

Support us

The Randa Meetings and other sprints indeed bring our software stack forward! Nowhere else such a nice group of KDE developers can meet up to solve such problems together! Please check out the fundraiser for the ongoing Randa Meetings.

Working on KDevelop and KDE on Windows in Randa

June 13, 2016 | Tags: kde, kdevelop, kde-windows, kderanda16

Right now, around 40 developers are working together on bringing KDE to other platforms in Randa (Switzerland) for an entire week.

I just arrived Sunday evening, and we immediately started with discussions around KDE on different platforms. Not much coding has happened yesterday evening yet, but I at least managed to work-around a compiler bug of GCC 4.8 showing up in the KDevelop code base.

My plans for this week are as follows:

Preparing the KDevelop 5.0 release (fixing release blockers)
Introduce the Windows CI for KDE (means: adding a Jenkins Windows slave to build.kde.org, making it produce useful results)
Generally be a source of help for people trying to port their application to Windows
Plus upstreaming patches I've been too lazy to fix up and push yet (hint: https://git.reviewboard.kde.org/r/124905/, etc.)

If you'd like to see what all the KDE people are working on, here's a work log of all participants of the sprint:
https://community.kde.org/Sprints/Randa/2016/Work

If you want to support us, please donate to help us keeping up these developer meetings!

Thanks!
Kevin

Building KDevelop 5 from source on Ubuntu 15.10

February 16, 2016 | Tags: kdevelop, kde, howto, ubuntu, debian

Note: These instructions apply for Ubuntu 16.04, too

Recently, I've got several mails from people desperately trying to compile KDevelop on Ubuntu-based distros. Let's give them a hand by providing them a quick start guide!

This is a brief version of what's in https://community.kde.org/Frameworks/Building (which really needs a major overhaul at this point -- lots of outdated or redundant information in there. I also had my trouble setting this up from scratch, believe me or not)

I've tested this HOWTO on a fresh Ubuntu 15.10 VM.

Setup

Disclaimer: This HOWTO will just compile the bare minimum, we're not going to compile either of Qt5, KF5 or LLVM/Clang. We'll happily chose the distro package of a needed dependency when available.

Install required dependencies

Start a terminal (konsole, xterm, your choice)

$ sudo apt-get build-dep qtbase5-dev

$ sudo apt-get install libbz2-dev libxslt-dev libxml2-dev shared-mime-info oxygen-icon-theme libgif-dev libvlc-dev libvlccore-dev doxygen gperf bzr libxapian-dev fontforge libgcrypt20-dev libattr1-dev network-manager-dev libgtk-3-dev xsltproc xserver-xorg-dev xserver-xorg-input-synaptics-dev libpwquality-dev modemmanager-dev libxcb-keysyms1-dev libepoxy-dev libpolkit-agent-1-dev libnm-util-dev libnm-glib-dev libegl1-mesa-dev libxcb-xkb-dev libqt5x11extras5-dev libwww-perl libxml-parser-perl libjson-perl libboost-dev libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev libarchive-dev liblmdb-dev cmake git extra-cmake-modules "libkf5.*-dev" llvm libclang-dev

Git remote prefix

Let's setup a "kde:" prefix for git commands. Add the following text to your ~/.gitconfig:

[url "git://anongit.kde.org/"]
   insteadOf = kde:
[url "ssh://git@git.kde.org/"]
   pushInsteadOf = kde:

Install kdesrc-build

kdesrc-build is, simply put, a user-space package manager, which compiles KDE-related projects from source and installs them into a designated directory.

Let's set up kdesrc-build to install KDevelop into our $HOME:

mkdir ~/kdesrc  
cd ~/kdesrc  
git clone kde:kdesrc-build  
cd kdesrc-build  
cp kdesrc-buildrc-kf5-sample ~/.kdesrc-buildrc

# Install a symlink of kdesrc-build to a location in PATH
mkdir ~/bin  
ln -s $PWD/kdesrc-build ~/bin  
export PATH=~/bin:$PATH

You should append the line export PATH=~/bin:$PATH to ~/.bashrc so kdesrc-build is available in PATH everytime you open a terminal.

Configure kdesrc-build

edit ~/.kdesrc-buildrc

Replace /path/to/kdesrc-build/kf5-qt5-build-include with ~/kdesrc/kdesrc-build/kf5-qt5-build-include in that file

Add ignore-kde-structure true and make-options -jN to the global section in ~/.kdesrc-buildrc (this will make your life easier...):

global  
  ...
  ignore-kde-structure true
  make-options -j5 # NOTE: 5 is the number of jobs, this should usually be (number-of-cpu-cores + 1)
  ...
end global

Installing KDevelop and dependencies

Let kdesrc-build handle the compilation + installation of KDevelop and its (direct) dependencies

$ kdesrc-build --debug libkomparediff2 grantlee kdevplatform kdevelop-pg-qt kdevelop

The --debug parameter will give you the verbose output, all command invocations and compiler output. Helpful for trouble-shooting.

Note: If you ever want to update+recompile your complete KDevelop checkout(s), you simply run above command again (it'll reuse your old build information, so it'll just recompile the bare minimum)

Setting up a script for preparing the environment

Copy these commands to a new file called ~/.env-kf5:

export KF5=~/kde-5  
export QTDIR=/usr  
export CMAKE_PREFIX_PATH=$KF5:$CMAKE_PREFIX_PATH  
export XDG_DATA_DIRS=$KF5/share:$XDG_DATA_DIRS:/usr/share  
export XDG_CONFIG_DIRS=$KF5/etc/xdg:$XDG_CONFIG_DIRS:/etc/xdg  
export PATH=$KF5/bin:$QTDIR/bin:$PATH  
export QT_PLUGIN_PATH=$KF5/lib/plugins:$KF5/lib64/plugins:$KF5/lib/x86_64-linux-gnu/plugins:$QTDIR/plugins:$QT_PLUGIN_PATH  
#   (lib64 instead of lib, on OpenSUSE and similar)
export QML2_IMPORT_PATH=$KF5/lib/qml:$KF5/lib64/qml:$KF5/lib/x86_64-linux-gnu/qml:$QTDIR/qml  
export QML_IMPORT_PATH=$QML2_IMPORT_PATH  
export KDE_SESSION_VERSION=5  
export KDE_FULL_SESSION=true

Small note: If you're running a 32 bit kernel, replace lib64 with lib32 and x86_64-linux-gnu with i386-linux-gnu in above script

Running KDevelop

Every time you want to use your self-compiled KDevelop, you simply spawn a new terminal, and then run:

$ source ~/.env-kf5
$ kdevelop

That's it already! You should have a working version of KDevelop 5 running now!

Hacking on KDevelop

Enter the source directory, edit files (of course you can do that by importing ~/kdesrc/kdevelop into KDevelop, too!

$ cd ~/kdesrc/kdevelop
<edit files>

Now, to recompile kdevelop, just invoke kdesrc-build again:

$ kdesrc-build --debug kdevelop

OR just go to the build directory and invoke:

$ cd ~/kdesrc/build/kdevelop
$ make install

Restart KDevelop:

$ kdevelop

The same procedure applies for the kdevplatform repository -- in case you need to work on that part of KDevelop:

$ cd ~/kdesrc/kdevplatform
<edit files>  
...

You're ready to contribute your patch now!

Contributing patches

The recommended way to contribute patches it to post them to KDE's Phabricator instance. The easiest way to create patches is to use Phabricator's Arcanist command-line tool.

The very brief version of what you have to do is:

$ cd ~/kdesrc/kdevelop
<edit files>  
$ arc diff
<arc will guide you through the required steps>

See here for more details: https://techbase.kde.org/Development/Phabricator#Using_Arcanist (in case you're not familiar with Arcanist at all)

Troubleshooting

Problems with kdesrc-build

In case kdesrc-build fails it will usually print a few lines like this at the end of the run:

<<<  PACKAGES FAILED TO BUILD  >>>  
libkomparediff2 - ~/kdesrc/log/2016-02-16-07/libkomparediff2/cmake.log  
:-(

Inspect that log to figure out what's going on:

$ cat ~/kdesrc/log/2016-02-16-07/libkomparediff2/cmake.log
CMake Error at CMakeLists.txt:5 (find_package):  
  Could not find a package configuration file provided by "ECM" (requested
  version 0.0.9) with any of the following names:

    ECMConfig.cmake
    ecm-config.cmake

  Add the installation prefix of "ECM" to CMAKE_PREFIX_PATH or set "ECM_DIR"
  to a directory containing one of the above files.  If "ECM" provides a
  separate development package or SDK, be sure it has been installed.

In this case: the ECM (extra cmake modules) package is missing. The way you usually fix these kind of problems is to head over to http://packages.ubuntu.com and search for the distro package providing a particular file (ECMConfig.cmake in this case).

So the package search reveals extra-cmake-modules being a hot candidate; to fix above error we simply install the package and the restart the build:

$ apt-get install extra-cmake-modules
<restart build>  
$ kdesrc-build ...

The error should be gone now.

Help

We're highly active in IRC, feel free to join us by visiting #kdevelop on Freenode. A web-based client can be found here: https://kiwiirc.com/client/irc.freenode.org/kdevelop

Just contact one of the core developers with the nick names milian, scummos, apol or kfunk if you need help.

The other way to get in touch is to write a mail to kdevelop-devel@kde.org

See here for details on how to reach us: https://www.kdevelop.org/contribute-kdevelop

We're always trying to be as helpful as possible!

Enjoy!

Icemon 3.0.1 release

February 8, 2016 | Tags: icemon, kde, icecc

Exactly one year after the 3.0 release, here's the next minor release of the Icemon 3.x series.

No major changes this time, merely bug fixes and small code refactorings.

Here's the changelog for the 3.0.1 release:

Bug fixes:

- Added work-around for build for icecc.a using old CXXABI (#24)
- Fixed build with Qt 5.5 
- Improved how docbook2man is searched for (PR #21)

Internal Changes:

- Added Doxygen support to CMake
- Modernized CMake code (FindIcecream.cmake, etc.)
- Modernized source code to use C++11 features (override, nullptr, auto)

Get it:

$ git clone https://github.com/icecc/icemon 
$ git checkout v3.0.1

Enjoy!

Icemon 3.0 release

February 8, 2015 | Tags: kde, icemon, icecc

It is my pleasure to finally release Icemon 3.0 to the public. If you don't know it -- Icemon is a GUI monitor for Icecream, a distributed compiler system.

Debian/Ubuntu have been packaging the Git version for quite a while now, and certain packagers kept nagging me for a release of a newer version.

So, what happened over the last years? Actually, first of all, I took over maintainership over that little tool, and cleaned up and improved code (and coding style, ugh) where I could. What we have now is a kdelibs-free, Qt5 only, C++11-enabled Icemon with proper model/view architecture that enables us to implement QtQuick2-based views.

I know that quite a few KDE people use it -- have fun with it!

Screenshot

Screenshot: Icemon Star View

Changelog

Features:

Star View got a fresh look, improved Detailed Host View (https://github.com/icecc/icemon/pull/17)
Added simple man page (https://github.com/icecc/icemon/pull/9)
Starting a change log

Bugfixes:

Fix RPATH issues (https://github.com/icecc/icemon/pull/18)
Lots of other small bug fixes in the Icemon views

Removals:

Dropped the "Pool View", which was never fully implemented

Internal Changes:

Ported icemon to Qt5
No longer depends on kdelibs
Better separation of model/view throughout the source code
- Enables possibility to write QtQuick-based views now

New porting helper: convert-to-cmake-automoc.pl

January 11, 2015 | Tags: kde, porting, snippets, cmake, automoc

Disclaimer

First time I wrote more than a one-liner in Perl, the Swiss Army chainsaw of scripting languages. After quickly skimming the beginner's tutorial, (and no longer just copying Perl one-liners from Stackoverflow and friends), I consider myself ready to write porting helpers for kde-dev-scripts.git! Hey!

If you still don't know those scripts, check them out now. They're tremendously helpful, especially those for porting KDE4-based applications to Qt5/KF5. I wouldn't want to miss them!

Make the most out of CMake's Automoc

Now there's a new porting helper: convert-to-cmake-automoc.pl, just committed today with https://git.reviewboard.kde.org/r/121991/.

This script tries to remove includes such as #include "<basename>.moc" from cpp files to make source code suitable for CMake's Automoc feature.

There are some pitfalls, though:

In some cases, one still needs to #include "<basename>.moc"; for example when the moc-generated code uses classes declared inside the .cpp file (hint: K_PLUGIN_FACTORY) -- in this case the moc file cannot be compiled in a separate translation unit.
In some cases, one still needs to #include "moc_<basename>.cpp"; for example when Q_PRIVATE_SLOT is used in the header file

This script handles these cases.

Example: attica.git

Invoking this script on your source code will fix the annoying "No output generated" warnings from moc:

$ make
...
/home/kfunk/devel/src/kf5/frameworks/attica/tests/projecttest/projecttest.cpp:0: Note: No relevant classes found. No output generated.
...

Now run find -iname "*.cpp" | xargs convert-to-cmake-automoc.pl and you'll end up with this patch.

Scripting GDB to execute commands at particular breakpoints

August 29, 2014 | Tags: kde, kdevelop, gdb, gdb scripting

This might be old news for the more experienced programmers out there, but yes, we can script GDB to do $stuff whenever it hits a breakpoint. With GDB's logging to file feature this can be super handy when trying to get a backlog of backtraces whenever a certain event arises.

Example use-case

Let's consider the following problem we'd like to debug: In KDevelop (Frameworks branch) we always got this annoying warning from Qt when exiting the application:

Output: QMutex: destroying locked mutex

Now, we can easily find out by grepping the Qt code base that this message is printed in qmutex.cpp:201 (which is inside ~QMutex). So, in order to figure out who's calling the destructor of QMutex and causing this output, let's put a breakpoint on qmutex.cpp:201 and re-run KDevelop and try to close it.

(gdb) break qmutex.cpp:201
Breakpoint 1 at 0x7ffff58f04bf: file /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp, line 201.

This leads to the following backtrace:

Breakpoint 1, QMutex::~QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, __in_chrg=) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:201
201         qWarning("QMutex: destroying locked mutex");
#0  QMutex::~QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, __in_chrg=) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:201
#1  0x00007ffff51638aa in __cxa_finalize (d=0x7ffff3428b78) at cxa_finalize.c:56
#2  0x00007ffff33f1573 in __do_global_dtors_aux () from /home/krf/devel/install/kf5/lib/x86_64-linux-gnu/libKDevPlatformUtil.so.9
#3  0x00007fffffffd830 in ?? ()
#4  0x00007ffff7dea73a in _dl_fini () at dl-fini.c:252

Unfortunately, the QMutex is destroyed during static deinitialization (notice the __do_global_dtors_aux call in the backtrace). Now, due to backtrace, we still don't know which QMutex in our code base got destroyed while being locked. We see that it is being statically initialized and must come out of libKDevPlatformUtil.so, but nothing more.

Problem: How do we find out which QMutex this was? Well, we need to check where this particular QMutex was first constructed.

GDB scripting to the rescue

I'd now like to print a backtrace each time we encounter the QMutex constructor (thus, QMutex::QMutex)

(gdb) break QMutex::QMutex
Breakpoint 2 at 0x7ffff58f040e: file /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp, line 178.

Additionally, I want to print a backtrace each time the breakpoint is encountered:

(gdb) command 2
Type commands for breakpoint(s) 2, one per line.
End with a line saying just "end".
>backtrace 10
>continue
>end

The command function makes GDB do the following each time it hits breakpoint 2: Print a backtrace limited to 10 frames and continue. (You can put whatever you need inside the command/end block.)

Furthermore, I'd like to get this logged to a file:

(gdb) set logging file gdb.txt
(gdb) set logging on
Copying output to gdb.txt.
(gdb) set pagination off

Now, let's restart KDevelop and close it again

(gdb) run

We'll again hit the breakpoint when printing the QMutex warning when static deinitialization happens:

Breakpoint 1, QMutex::~QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, __in_chrg=) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:201
201         qWarning("QMutex: destroying locked mutex");
#0  QMutex::~QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, __in_chrg=) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:201
#1  0x00007ffff51638aa in __cxa_finalize (d=0x7ffff3428b78) at cxa_finalize.c:56
#2  0x00007ffff33f1573 in __do_global_dtors_aux () from /home/krf/devel/install/kf5/lib/x86_64-linux-gnu/libKDevPlatformUtil.so.9
#3  0x00007fffffffd830 in ?? ()
#4  0x00007ffff7dea73a in _dl_fini () at dl-fini.c:252

Duly note the this pointer of the QMutex destroyed from the backtrace (QMutex::~QMutex (this=0x7ffff3428ba0 ...): It's 0x7ffff3428ba0

Note that in gdb.txt we now have the following contents (some parts replaced by ... for increased readability):

(...)

Breakpoint 2, QMutex::QMutex (this=0x7ffff7dd8b78 <(anonymous namespace)::resInit+24>, mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
178 QMutex::QMutex(RecursionMode mode)
#0  QMutex::QMutex (this=0x7ffff7dd8b78 <(anonymous namespace)::resInit+24>, mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
#1  0x00007ffff7be0e29 in (anonymous namespace)::ResInitUsage::ResInitUsage (this=0x7ffff7dd8b60 <(anonymous namespace)::resInit>) at /home/krf/devel/src/kf5/frameworks/kdelibs4support/src/kdecore/k3resolvermanager.cpp:166
#2  0x00007ffff7be2067 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /home/krf/devel/src/kf5/frameworks/kdelibs4support/src/kdecore/k3resolvermanager.cpp:237
#3  0x00007ffff7be2096 in _GLOBAL__sub_I_k3resolvermanager.cpp(void) () at /home/krf/devel/src/kf5/frameworks/kdelibs4support/src/kdecore/k3resolvermanager.cpp:815
#4  0x00007ffff7dea13a in call_init (...) at dl-init.c:78
#5  0x00007ffff7dea223 in call_init (...) at dl-init.c:36
#6  _dl_init (...) at dl-init.c:126
#7  0x00007ffff7ddb30a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#8  0x0000000000000003 in ?? ()
#9  0x00007fffffffde39 in ?? ()

Breakpoint 2, QMutex::QMutex (this=0x7ffff7dd8b98 , mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
178 QMutex::QMutex(RecursionMode mode)
#0  QMutex::QMutex (this=0x7ffff7dd8b98 , mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
#1  0x00007ffff7be68fe in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /home/krf/devel/src/kf5/frameworks/kdelibs4support/src/kdecore/k3resolverstandardworkers.cpp:97
#2  0x00007ffff7be6956 in _GLOBAL__sub_I_k3resolverstandardworkers.cpp(void) () at /home/krf/devel/src/kf5/frameworks/kdelibs4support/src/kdecore/k3resolverstandardworkers.cpp:1049
#3  0x00007ffff7dea13a in call_init (...) at dl-init.c:78
#4  0x00007ffff7dea223 in call_init (...) at dl-init.c:36
#5  _dl_init (...) at dl-init.c:126
#6  0x00007ffff7ddb30a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#7  0x0000000000000003 in ?? ()
#8  0x00007fffffffde39 in ?? ()
#9  0x00007fffffffde62 in ?? ()

Breakpoint 2, QMutex::QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
178 QMutex::QMutex(RecursionMode mode)
#0  QMutex::QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
#1  0x00007ffff33f23ba in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /home/krf/devel/src/kf5/extragear/kdevelop/kdevplatform/util/foregroundlock.cpp:29
#2  0x00007ffff33f24ab in _GLOBAL__sub_I_foregroundlock.cpp(void) () at /home/krf/devel/src/kf5/extragear/kdevelop/kdevplatform/util/foregroundlock.cpp:235
#3  0x00007ffff7dea13a in call_init (...) at dl-init.c:78
#4  0x00007ffff7dea223 in call_init (...) at dl-init.c:36
#5  _dl_init (...) at dl-init.c:126
#6  0x00007ffff7ddb30a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#7  0x0000000000000003 in ?? ()
#8  0x00007fffffffde39 in ?? ()
#9  0x00007fffffffde62 in ?? ()

(...a lot more...)

Every time QMutex::QMutex was encountered, GDB printed a backtrace and logged it to the file.

Now, in order to find out where the QMutex comes from we simply search the string 0x7ffff3428ba0 inside gdb.txt and we'll find:

Breakpoint 2, QMutex::QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
178 QMutex::QMutex(RecursionMode mode)
#0  QMutex::QMutex (this=0x7ffff3428ba0 <(anonymous namespace)::internalMutex>, mode=QMutex::NonRecursive) at /home/krf/devel/src/qt5/qtbase/src/corelib/thread/qmutex.cpp:178
#1  0x00007ffff33f23ba in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /home/krf/devel/src/kf5/extragear/kdevelop/kdevplatform/util/foregroundlock.cpp:29
#2  0x00007ffff33f24ab in _GLOBAL__sub_I_foregroundlock.cpp(void) () at /home/krf/devel/src/kf5/extragear/kdevelop/kdevplatform/util/foregroundlock.cpp:235
#3  0x00007ffff7dea13a in call_init (...) at dl-init.c:78
#4  0x00007ffff7dea223 in call_init (...) at dl-init.c:36
#5  _dl_init (...) at dl-init.c:126
#6  0x00007ffff7ddb30a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2

Frame 2 shows: This mutex comes from /home/krf/devel/src/kf5/extragear/kdevelop/kdevplatform/util/foregroundlock.cpp:29, which says QMutex internalMutex;

We've found it!

At this point we can finally start solving our original problem of the destruction of a locked mutex, because now we at least know which mutex is causing this.

Other use-cases

Tracing ref-counting issues

You know that some object (for example QCoreApplication in Qt5) has a refcount higher than zero when exiting the application, but you don't know which object is still keeping a reference on it.

How to debug: Print backtraces each time we call the hypothetical ref() and deref() (for example QCoreApplication::{de}ref()). Now simply check which object never calls deref() in the GDB output file.

Verdict

GDB's scripting capabilities can be tremendously useful when attempting to debug issues where the backtrace at the point of crash or some other event just isn't enough.

This helped me to fix several issues in KDevelop already, which would have been hard to tackle otherwise.

Also see: https://sourceware.org/gdb/current/onlinedocs/gdb/Break-Commands.html

Randa Report: Hacking on KDE and meeting friends

August 17, 2014 | Tags: kde, kdevelop, clang, gsoc, kde sdk, randa

Hey there,

I'm already back home and now like to you let you know what I've been doing the last week during the Randa Sprint in the Swiss Alps.

Quick summay: It has been an immense event!

View from our hacking room, in Randa, Switzerland

Last week I've been mostly occupied with porting KDevelop to KF5 and, part of that, succeeded in making it compile and run on Windows as well. In between the hacking sessions, we had a lot of fruitful discussions which concerned the future of KDevelop and the whole KDE SDK itself. Let me break that down into the individiual tasks accomplished throughout the week.

Report

Day 1 (Sat, 9th August)

Arriving to Randa, Switzerland at around afternoon. Meetup with all the KDE fellows scattered all around in the house.

Clang integration: C++ Macro navigation support

In the afternoon I was working a bit on tasks from kdev-clang (my GSoC project this year), which I wanted to get fixed first. So, I managed to bring back basic C++ Macro navigation support in our plugin (I'll explain the implementation details / shortcomings in another blog post).

KDevelop screenshot — KDevelop showing the uses of a macro definition when hovering the macro identifier and pressing "Show Uses"

Resolving inter-library dependencies in kdevplatform

Another largish patch landed in kdevplatform.git this evening: Splitting up the huge kdevplatformlanguage library (which contains all the language agnostic Definition-Use-Chain logic) into more managable parts. I've factored out the part that provides the infrastructure for serializing the DUChain items, and created kdevplatformserialization, which now contains all the classes required for writing your own (serializable) item repository.

This change resolved a few issues with inter-library dependencies we had in kdevplatform. This also helped making kdevplatform compile on Windows in the end.

Day 2 (Sun, 10th August)

More porting of kdevplatform to KDE Frameworks. Mostly fixing up CMake-related code in order to be able to compile tests using new CMake macros (ecm_add_test). Pushing a few compilation fixes for Windows for both GCC and MSVC.

Day 3 (Mon, 11th August)

Fixing up unit tests in kdevelop/kdevplatform using the frameworks branch. Also fixing a few crashes that popped due to changed behavior in Qt5 (mostly event-handling related).

Day 4 (Tue, 12th August)

Switch Declaration/Definition feature in kdevplatform

Moving support of the "Switch Declaration/Definition" feature (something you can trigger via the menu or via the "Ctrl+Shift+C" shortcut) to kdevplatform. That in turn means, that any language (well, any language which disambiguates between definitions and declarations) can make use of this feature without further work. Of course, the main motivation was to get this working for kdev-clang. Review-request here: https://git.reviewboard.kde.org/r/119648/

Basic Objective-C support in kdev-clang

Later that night, Alex Fiestas and me got into philosophical discussions about the Clang integration in KDevelop and suddenly the question about Objective-C support popped up. Realizing that we didn't ever look into that (despite being very well aware that Clang supports it), I decided to spent an hour on it in kdev-clang to see some first-hand results.

You can see the patch here. As you can see, this is mostly about making our Clang integration aware of Objective-C entities in the Clang AST. It's that trivial.

And here the result:

Note: If someone is interested in driving this further, that'd be greatly appreciated. Personally I won't have time in the near future to extend the Objective-C support (also, I don't do Objective-C these days, so I don't have a use for it)

Day 5 (Wed, 13th August)

Fixing up the Grep Dialog in KDevelop (frameworks branch). There were some issues with event-handling in KComboBox and default selection inside the dialog button box. In the end, I decided to port this over to QDialog and QComboBox right away and fixed up both issues during that.

Another major issue for KDevelop on Windows got fixed this day: Windows path handling in our custom QUrl-replacement class KDevelop::Path: We now also support Window's single-drive letter based file schemes (e.g. C:\tmp) here. That fixed include path / include lookup on Windows.

Day 6 (Thu, 14th August)

Attempting to fix hang-on-exit issue in KDevelop

This day, I was mostly spending (read: wasting) time attempting to fix the most apparent issue in KDevelop (frameworks branch): KDevelop not exiting cleanly when asked to be shutdown. I'm still not exactly sure what's wrong, but it seems like some object is not calling QCoreApplicationPrivate::deref, and hence the event loop is not being quit when the last window is closed (because QCA still assumes to be in use, i.e. the refcount being non-zero)

tl;dr: I'll keep you posted as soon as I find out what's wrong here.

Daytrip time

Thursday afternoon a whole bunch of the KDE fellows made a great day trip to get a closer look at the wonderful Matterhorn. For this, we got to Randa by taxi and got the chance to walk around in the (admittedly very touristy) town Zermatt. After a few minutes of walk, we got to see this amazing view of the Matterhorn:

View to Matterhorn from Zermatt, Switzerland

Day 7 (Fri, 15th August)

After a good week of productive hacking and meeting friends in the Swiss Alps, I left Randa very early in the morning by train towards Zurich, for my flight back to Berlin.

Discussions

It has been a highly productive week this time. The team had a lot to discuss about future ideas that concern both KDevelop and the KDE SDK as a whole.

Interesting topics we've mentioned, which were directly related to KDevelop:

Improving the dashboards inside KDevelop: We'd like to introduce a session-specific dashboard that shows information about the currently loaded projects, such as recent commits, recent bug entries, latest mailing list entries, etc.

Reducing the ramp-up time needed for newcomers: We want to make it easier to download/build/run applications, and make it easier to contribute back patches to reviewboard for newcomers. Part of that we'd like make it easier to fetch dependencies of a particular KDE project, so the developer really doesn't need to worry too much about setting up his/her environment. We (the KDE SDK team) planned to improve kdesrc-build in a way it can be used as a backend for handling all this.

We also had a bunch of non-KDevelop related discussions, let me briefly summarize them:

The KF5 book is coming along nicely, and lots of people have been involved into making it shine
The KDE apidocs site got some love and looks much better now
(a lot more)

Summary

Thanks for making the event happen, thanks to all the donors for this year's Randa fundraiser! As always, it's been an terrific event, a super productive week altogether.

Thanks a lot to Mario Fux and his family/friends for organizing and keeping us happy during the event.

GSoC Status Report: Code Completion features

July 30, 2014 | Tags: kde, kdevelop, clang, gsoc

Context: I'm currently working on getting Clang integration in KDevelop into usable shape as part of this year's Google Summer of Code. See the initial blog post for my road map.

While we had basic support for code completion provided by Clang from the beginning (thanks to David Stevens for the initial) work, it still didn't really feel that useful in most cases. In the last two weeks I've spent my time streamlining the code completion features provided by Clang.

This blog post is going to be full of screenshots showing the various features we've been working on lately.

Task recap: Code completion

Code completion: Implement “virtual override completion”. Also the automatic replacement of . to -> and vice-versa in case of pointer/non-pointer types is still missing.
“Implement function” feature: If a function is only declared but not defined, we offer a "implement function" helper item in the code completion. This is currently not yet ported to clangcpp.
"Switch to Definition/Declaration” feature: If the cursor is at some declaration in a header file, KDevelop offers a shortcut to automatically switch to its definition in the source file (opening the corresponding file in the active view). This is not yet possible in clangcpp.
Show viable expressions for current context: When inside a function call or constructor call, show viable expressions which fit the signature of the current argument. Example: int i = 1; char* str = “c”; strlen( – this should show variable str in the completion popup as best match.
Include completion: Oldcpp offers completion hints when attempting to #include some file, port this to clangcpp.

Achievements

Virtual override completion

Simple case

When in a class context, we can now show completion items for methods that are declared virtual in the base class.

By pressing Enter now, KDevelop automatically inserts the following code at the current cursor position:

virtual void foo()

Oh, no! Templates!

We've spent a bit of work to make this feature work with templated base classes, too. Have a look at this:

Nice, right?

Implement function helper

When encountering an undefined method which is reachable from within the current context, KDevelop offers to implement those via a tooltip

By pressing Enter now, KDevelop automatically inserts the following code at the current cursor position:

void Foo::foo()
{
}

This works for all types of functions, be it class member functions or free member functions and/or functions in namespaces. Since this is mostly the same code path as the "virtual override helper" feature, this plays nicely with templated functions, too.

"Switch to Definition/Declaration” feature

Sorry, no pictures here, but be assured: It works!

Pressing Ctrl+, ("Jump to Definition") while having the cursor on some declaration will bring you to the definition. Consecutively, pressing Ctrl+. ("Jump to Declaration") on some definition will bring you to the declaration of that definition.

Show viable expressions for current context

Best matches

This is some of the features we actually get for free when using Clang. We get the completion results by invoking clang_codeCompleteAt(...) on the current translation unit and iterating through the results libclang is offering us. Clang gives highly useful completion results, the LLVM team did an amazing job here.

Another example: Enum-case completion

You can play around with Clang's code completion ability from the command-line. Consider the following code in some file test.cpp:

enum SomeEnum { aaa, bbb };

int main()
{
    SomeEnum e;
    switch (e) {
    case 
    //   ^- cursor here
    }
}

Now do clang++ -cc1 -x c++ -fsyntax-only -code-completion-at -:7:9 - < test.cpp and you'll get:

COMPLETION: aaa : [#SomeEnum#]aaa  
COMPLETION: bbb : [#SomeEnum#]bbb

Awesome, right?

Issues: Too many code completion result from Clang

One thing I've found a bit annoying about the results we're getting is that Clang also proposes to explicitly call the constructors/destructors or assignment operators in some cases. Or in other words: It proposes too many items

Consider the following code snippet:

struct S
{
    void foo();
};

int main()
{
    S s;
    s. 
    //^- cursor here
}

Now doing clang++ -cc1 -x c++ -fsyntax-only -code-completion-at -:8:7 - < test.cc results in:

COMPLETION: foo : [#void#]foo()  
COMPLETION: operator= : [#S &#]operator=(<#const S &#>)  
COMPLETION: S : S::  
COMPLETION: ~S : [#void#]~S()

Using one of the last three completion results would encourage Clang to generate code such as s.S, s.~S or s.operator=. While these constructs point to valid symbols, this is likely undesired.
Solution: We filter out everything that looks like a constructor, destructor or operator declaration by hand.

So, in fact, what we end up showing the user inside KDevelop is:

Just what you'd expect.

Wrap-Up

Code completion features are mostly done (at least from the point-of-view of what Clang can give us here).

Still, there other interesting completion helpers that could^Wshould be ported over from oldcpp to kdev-clang, such as Olivier's lookahead-completion feature (which I find quite handy). This is not yet done.

I'm writing up yet another blog post which is going to highlight some of the other bits and pieces I've been busy with during the last weeks.

Thanks!