March and April in KDE PIM

Following the post about what happened in KDE PIM in January and February let’s look into what the KDE PIM community has been up to in March and April. In total 38 contributors have made almost 1700 changes. Big thanks to everyone who helped us make Kontact better!

April 2020 Release

A new bundle of KDE applications has been released in April, including Kontact with its many bugfixes and improvements.

KDE PIM Sprint

Every year in April the PIM team meets in Toulouse in France for a weekend of discussions and hacking. This year due to the coronavirus it wasn’t possible for us to meet so instead we held a virtual KDE PIM Sprint. You can read the sprint agenda as well as Volker’s report from the sprint.

To highlight some of the topics discussed

  • Moving KDAV into KDE Frameworks
  • KParts usage in Kontact
  • Remove the Kolab Resource, encourage users to use IMAP+DAV instead
  • Data provider plugins for the KCalendarCore library
  • Porting Account Wizard away from Kross (port to QML, eventually replace it completely by KAccounts)

KMail

Bugfixes

KMail has received its usual dose of bugfixes, mostly those:

  • Fixed a crash when sending an email (Laurent Montel, bug 419473)
  • Fixed a crash when adding too many recipients (Jonathan Marten, D28876)
  • Fixed a bug when KMail showed only a part of an HTML email (Laurent Montel, bug 419949)
  • Fixed a bug when KMail did not correctly display email answer in HTML mode (Laurent Montel, bug 418482)
  • Fixed broken message status filter (Laurent Montel, bug 419720)
  • Fixed the maildir backend creating junk folders around and not storing the path configuration properly (Igor Poboiko, D27722, D27906 and D28634)
  • Fixed a crash when configuring the POP3 resource (Daniel Vrátil, bug 419726)
  • Fixed name of the top-level folder created by the EWS resource (Igor Poboiko, bug 416663)
  • Fixed the EWS resource not storing user password properly (Igor Poboiko, bug 414789)

Improvements

There were some exciting improvements to KMail as well: Sandro Knauß has implemented support for Protected Headers for Cryptographic E-Mails. This means that we also send a signed/encrypted copy of headers and display the signed/encrypted headers if available and ignores the unsecure headers. Currently we don’t obfuscate the subject to not break current workflows. Those things will be improved later on. Sandro together with Thomas Pfeiffer get a funding from nlnet to improve mail encryption. That means there will be more improvement happen the next months. The next topic they will look at is to add Autocrypt support for KMail.

Volker has improved the look and feel of the “HTML Content” and “External References” warnings in emails.

Improved look and feel of the "HTML Content" and "External References" warnings

As the Libre Avatar service has come back from the dead a while ago, so did now the support for it in KMail. The ‘Export to PDF’ feature which we introduced in the previous report has been polished (Daniel Vrátil, D27793).

The ‘Move To Trash’ code has been optimized so that deleting large amounts of emails should now be faster.

For developers it is now possible to open Chromium DevTools inside the message viewer pane to make it easier to debug message templates.

KOrganizer

Bugfixes

  • Fixed crashes in category and filter managers (Allen Winter)
  • Fixed bug when single instance of a recurring event couldn’t be changed (Shashwat Jolly, bug 410758)
  • Fixed crash when creating a new Todo from Kontact (David Faure, bug 420192)
  • Fixed ‘Only resources can modify remote identifiers’ error when re-editing event (Igor Poboiko, bug 407965)
  • Fixed the DAV resource getting stuck when parse error occurs (David Faure, D27858)

Improvements

The Google Calendar and Google Contacts backends have been merged into a single Google Groupware resource (Igor Poboiko, D28560). The change should be mostly transparent to users, the old backends will be migrated to the new unified backend automatically after update. During this Igor also fixed various bugs and issues in the backends and the LibKGAPI library, big kudos to him!

The DAV resource is now able to synchronize the calendar color from KOrganizer to the DAV server (David Faure, D28938). Related to that, the menu to configure calendar color in KOrganizer has been simplified by removing the “Disable Color” action.

It is now easier to recognize and set the default calendar and the event editor now respects the settings correctly.

It is now easier to see the default calendar in KOrganizer

KJots

KJots, the note taking application, which has been on life support for 5 years, has received some love recently thanks to Igor Poboiko. Most of the things were happening under the hood: some ancient dusty code has been dropped, some refactoring happening, etc. However, if you still use KJots, you might also notice quite a number of changes too. And if you don’t, it’s a good time to consider using it :)

Bugfixes

  • Fixed a data loss issue due to bugs in the Akonadi Maildir resource, which is used as a KJots backend
  • Fixed a crash on startup (bug 420228)
  • Fixed multiple actions for the same shortcut (bug 384000)
  • Fixed bookmarks support (D29073)
  • Fixed export to plain text and to HTML
  • Fixed random scrollback jumps (bug 195828)
  • Fixed nested bullet lists breaking the undo stack (bug 256001)

Improvements

  • Link destination is displayed in tooltip (D29289)
  • Ctrl+click follows the link (bug 244846)
  • Priting support has been revived (D29045)
  • The text editing widget now supports different headings (bug 230317)
  • Improved support for nested bullet lists

Future

Igor has quite huge plans for the future of KJots. First of all, more bug squashing. Secondly: ability to store notes in Markdown format, synchronization with online services (thoughts are on OwnCloud/NextCloud, or proprietary Evernote). On a lesser scale, the port to the same text editing component as used by KMail email composer is being considered, which will give KJots more text-editing features. There are also plans to add a support for inline checkboxes introduced in Qt 5.14, which would allow making checklists and TODO-lists in KJots, and ability to sort books and pages by their modification date (so more relevant would pop up first).

Other components

Other parts of PIM has also received bugfixes and improvements. Kleopatra, the certificate management software, now displays GPG configuration tabs and option groups always in the same order (Andrey Legayev, T6446). A bug in Akregator has been fixed that could have caused some feeds to have an icon missing (David Faure, D28581). KAlarm has received a bunch of UI improvements as well as some smaller features - for instance it is now possible to import alarms from multiple calendars at once and the calendar list is now sorted by name (all by David Jarvie).

Common Infrastructure

Lots of work went into modernizing Akonadi, the “backend” service for Kontact. One major change was switch to C++17 and some initial usage of C++17 features internally (public API is still C++11-compatible). Widgets for managing Tags have been improved and polished and the deprecated ItemModel and CollectionModel have been removed.

The KIMAP library has been optimized to better handle large message sets (Daniel Vrátil, D28944). The KLDAP library can now connect to LDAP server using SSL encryption (Tobias Junghans, D28915), alongside the existing TLS support.

Volker Krause has been working on preparing the KDAV library (which implements the DAV protocol) into KDE Frameworks.

Laurent Montel has been working throughout the entire PIM codebase, preparing it to port to Qt6, once it’s available.


Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

January and February in KDE PIM

Following the post about what happened in KDE PIM in November and December by Volker, let’s look into what the KDE PIM community has been up to in the first two months of the new year. In total 23 contributors have made 740 changes.

KMail

  • A warning is shown when message composer is opened by clicking on a URL that asks for a file to be attached

Files attached by clicking on a mailto URL

  • DKIM validation support is enabled by default now

DKIM validation information

  • The email can now be easily exported to PDF
  • Improved support for rendering markdown in emails
  • “Important email” icon is now used consistently in KMail
  • Fixed “Move Message to Folder” dialog not accepting certain characters (#415850)
  • Fixed large emails not being displayed (#387061)
  • Fixed attachments being lost when message is saved as draft (#416369)

KOrganizer

  • Fixed restoring last used calendar in incidence editor (#411191)
  • Fixed fallback to the default calendar in incidence editor
  • Fixed a crash in release builds (#417794)

Akregator

  • Fixed filename extensions being translated (#416983)

KAlarm

  • Fixed message captions
  • Added warning when no default alarm calendar is set during archiving
  • Fixed crashes
  • Better handling when calendars are added or removed

Common Infrastructure

Laurent has been working on porting our code away from API that has been deprecated in Qt 5.15. Volker has been working on removing KDBusConnectionPool from all of KDE PIM.

For network communication we now use some safer defaults - for example we enabled HSTS by default and we don’t allow redirects to less-safe protocols (i.e. from https:// to http://).

KMail and some other components have begun integrating KUserFeedback to provide some basic telemetry information about usage.


Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

Q_PRIVATE_SLOT with new connect syntax

When using PIMPL, we sometimes want to move implementation of slots into the private class as well. In order for Qt to be able to invoke those slots that formally exist only in the private class (which usually is not a QObject), we use the Q_PRIVATE_SLOT macro in the main class. It allows Qt to invoke the slot method, even though it exists in the private class.

Let’s have a short example:

/// mybutton.h

class MyButtonPrivate;
class MyButton : public QPushButton {
    Q_OBJECT
public:
    explicit MyButton(QWidget *parent);
    ~MyButton() noexcept override;

private:
    std::unique_ptr<MyButtonPrivate> const d_ptr;
    Q_DECLARE_PRIVATE(MyButton);

    Q_PRIVATE_SLOT(d_func(), void onClicked(bool));
};

/// mybutton.cpp

class MyButtonPrivate
{
public:
    void onClicked(bool clicked) {
        qDebug() << "Clicked!";
    }
};

MyButton::MyButton(QWidget *parent)
    : QPushButton(parent)
    , d_ptr(std::make_unique<MyButtonPrivate>())
{
    // Connecting to slot on "this" (MyButton*), although "onClicked" is defined in MyButtonPrivate
    connect(this, SIGNAL(clicked(bool)),
            this, SLOT(onClicked(bool)));
}

MyButton::~MyButton() noexcept = default;

Q_PRIVATE_SLOT does not create any new method in the MyButton class. The way Q_PRIVATE_SLOT works is that it just instructs moc to generate a metacall that looks like obj->d_func()->onClicked(val) instead of obj->onClicked(val), like it does for normal slots or invokables.

This approach has one big disadvantage: it means that Q_PRIVATE_SLOTs can only be invoked through the old QObject::connect() syntax.

So far I’ve been using a simple workaround to get all the compile-time checks that I would get with the new connect syntax normally:

connect(this, &MyButton::clicked,
        this, [this](bool clicked) { d_func()->onClicked(clicked); });

Here we use a lambda to forward the call to the actual PIMPL’ed slot. This is somewhat better than the old connect syntax but IMO it has two major drawbacks:

It’s hard to read - it’s difficult to immediatelly decipher what method the call is actually being forwarded to.

It’s tedious to write - it’s a lot of boilerplate code to be written and if there are too many arguments it becomes quite ugly. C++14 generic lambdas improve the situation a bit since we can use auto instead of spelling out the argument types, but I don’t think it makes the code necessarily better to read:

connect(this, &MyObject::mySignal,
        this, [this](const auto &foo, auto bar, auto *baz) {
            d_func()->mySlot(foo, bar, baz);
        });

It got me thinking if there might be some way to auto-generate the forwarding lambda and be able to just use the pointer-to-function here somehow.

In the end I came up with this tiny template function, which takes the d-pointer and the pointer to the PIMPL’ed slot and returns a generic lambda which gets passed into QObject::connect and which forwards arguments to the real slot method.

template<typename DPtr, typename Slot>
auto priv_slot(DPtr &&dptr, Slot &&slot)
{
    return [&dptr, &slot](auto && ... args)
    {
        (dptr->*slot)(std::forward<decltype(args)>(args) ...);
    }
}

The result has all the benefits of the new QObject::connect() syntax without the ugliness of the “forwarding lambda”:

connect(this, &MyObject::mySignal,
        this, priv_slot(d_func(), &MyObjectPrivate::onSignal));

I'm Going to Akademy!

I'm Going to Akademy!

In just five days I’ll be on my way to Akademy! I’m so excited to meet with all my friends from KDE! After missing the conference weekends in Almería and Vienna, I’ll be able to get the full Akademy experience again - including delivering a talk!

Build Expressive APIs with Modern C++

I’ll be giving a talk about how to use some cool features from C++17 (even if you cannot use C++17!) in your code to make it easier for others (and yourself) to understand the intentions of the code, which helps improve productivity and reduce bugs and errors. The talk will be on Sunday at 14:35 in room U4-08.

KDE PIM BoF

The KDE PIM team will have a BoF session on Monday morning (10:30 - 12:30) in room U1-04. If you want to talk to us about anything KDE PIM related, feel free to stop by!

Other than that my main intention is to make use of the whole week to do some intensive hacking on Akonadi, in-person debugging and fixing bugs :)

See you all in Milan!

Kontact and Google Integration Issues

Lately there were some issues with the Google integration in Kontact which caused that it is no longer possible to add new Google Calendar or Gmail account in Kontact because the log in process will fail. This is due to an oversight on our side which lead to Google blocking Kontact as it did not comply with Google’s policies. We are working on resolving the situation, but it will take a little bit.

Existing users should not be affected by this - if you already had Google Calendar or Gmail set up in Kontact, the sync should continue to work. It is only new accounts that cannot be created.

In case of Gmail the problem can mostly be worked around when setting up the IMAP account in KMail by selecting PLAIN authentication1 method in the Advanced tab and using your email and password. You may need to enable Less Secure Applications in your Google account settings in order to be able to log in with regular email address and password.

If you are interested in the technical background of this issue, the problem comes from Google’s OAuth App Verification process. When a developer wants to connect their app to a Google service they have to select which particular services their app needs access to, and sometimes even which data within each service they want to access. Google will then verify that the app is not trying to access any other data or that it is not misusing the data it has access to - this serves to protect Google users as they might sometimes approve apps that will access their calendars or emails with malicious intent without them realizing that.

When I registered Kontact I forgot to list some of the data points that Kontact needs access to. Google has noticed this after a while and asked us to clarify the missing bits. Unfortunately I wasn’t able to react within the given time limit and so Google has preemptively blocked login for all new users.

I’m working on clarifying the missing bits and having Google review the new information, so hopefuly the Google login should start working again soon.

  1. Despite its name, the PLAIN authentication method does not weaken the security. Your email and password are still sent safely encrypted over the internet. 

March and April in KDE PIM

The KDE PIM community has been working hard in March and April to fix various bugs and bring new features and improvements to Kontact, Itinerary and other KDE PIM projects. Here is a summary of some of the changes that landed in the past two months.

KDE PIM Sprint

The team has met for a weekend for a traditional spring PIM sprint in Toulouse at the beginning of April. We focused on solving some long-standing issues in Akonadi, prepared some of the KDE PIM libraries to be moved to KDE Frameworks and discussed outreach and how to attract more people to KDE PIM.

You can read David’s report from the sprint.


KMail

KMail has received a nice batch of bugfixes for the KDE Applications 19.04 release as well as some small features, which will be available in the 19.08 release.

Improvements and New Features

The email composer has received improvements to support UTF-8 emojis (Laurent Montel, KDE Applications 19.08) and to support composing rich-text emails using Markdown with syntax highlighting (Laurent Montel, KDE Applications 19.08).

Additionally, notification when sending email fails is now less intrusive (Laurent Montel, KDE Applications 19.08).

Bugfixes

  • Fixed crash on shutdown due to mishandling of QWebEngineProfile (Jonathan Marten, D19559)
  • Fixed reconnecting to IMAP server when connection is lost during login (David Faure, D20028)
  • Fixed identity not appearing after creation (Laurent Montel, bug 391631)
  • Fixed deleting spam messages (Laurent Montel, bug 406324)
  • Fixed custom header search query failing with inconclusive error (Laurent Montel, bug 405456)
  • Fixed trying to send a message without any recipients fails quitely (Laurent Montel, bug 402577)
  • Fixed a crash switching on an ad-blocked (Laurent Montel, bug 405434)
  • Fixed a crash when opening a vCard attachment (Laurent Montel, bug 405791)

KOrganizer

During the PIM sprint, Volker Krause did a major work on cleaning up KCalCore, a library that implements the iCal standard to store events and tasks information, so that the library can be moved to KDE Frameworks. You can read more about Volker’s effort in his blog post.

New Features and Improvements

It is now possible to move event from one calendar to another in KOrganizer by changing the calendar in the incidence editor (David Faure, KDE Applications 19.08).


KAddressBook

Volker also worked on preparing the KContacts library, a vCard standard implementation, to be moved to KDE Frameworks alongside KCalCore. You can read about this on his blog as well.

Improvements

As part of his code cleanup, Volker has removed the DataMatrix from contact display, so we only display the QR code now (Volker Krause, KDE Applications 19.08), and did massive clean up of the contact display code. When displaying contact address on a map, KAddressBook now defaults to OpenStreetMaps (Volker Krause, KDE Applications 19.08).

Thanks to Laurent Montel it is now also possible to send SMS messages from KAddressBook through KDE Connect (KDE Applications 19.08).

Bugfixes

  • Fixed dialing contact’s phone-number from KAddressBook through KDE Connect (Volker Krause, D20353)

Akregator

Bugfixes

  • Make the feed URI the baseUrl for article previews (Pierre Ducroquet, D19739)
  • Fixed Delete key not working after using search (Laurent Montel, bug 394946)

Common Infrastructure

Improvements

David Faure and I have spent large part of the PIM sprint investigating and digging into two major issues we currently have in Akonadi: database deadlocks and a bug known as “multiple merge candidates”. We were unable to determine the real cause for the “multiple merge candidates” bug, so it remains unfixed for now. As a result of digging through the code base, however, David has produced a set of patches to hugely improve handling of database deadlocks and transaction rollbacks in the Akonadi server. He also did improve the sync scheduling code. I have removed a large chunk of code by removing mostly unused code to handle Collection references and LDAP/Kolab contact merging.

New Features

  • LibKGAPI: support for Team Drives API for Google Drive (David Barchiesi, T10521)
  • KItinerary: support for parsing train booking from SNCF confirmation emails (Volker Krause, bug 404451)

Bugfixes

  • Akonadi: fixed sync getting stuck after failure (David Faure, D19487, bug 399167)
  • Akonadi: fixed race conditions in Attribute handling (David Faure, D19556, D19632)
  • Akonadi: fixed crash when in resources when handling a collection change (Daniel Vrátil, bug 403642)
  • Akonadi: fixed crash when an Akonadi client unexpectedly disconnects from the server (Filipe Azevedo, D19983)

Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

DrKonqi and QtWebEngine

Here’s a little tip how to get DrKonqi, the KDE crash handler to work in applications that use QtWebEngine.

If your application uses QtWebEngine, you probably noticed that DrKonqi doesn’t pop up when the program crashes. This is because QtWebEngine installs its own crash handler, overriding the one DrKonqi has set up.

The workaround is quite simple but is not trivial to find because all of it is undocumented (and not everyone wants to dig into Chromium code…). The trick is to add --disable-in-process-stack-traces to QTWEBENGINE_CHROMIUM_FLAGS environment variable before initializing QtWebEngine:

const auto chromiumFlags = qgetenv("QTWEBENGINE_CHROMIUM_FLAGS");
if (!chromiumFlags.contains("disable-in-process-stack-traces")) {
    qputenv("QTWEBENGINE_CHROMIUM_FLAGS", chromiumFlags + " --disable-in-process-stack-traces");
}
...
auto view = new QtWebEngineView(this);
...

Here’s a full example of how we fixed this in Kontact

Kontact ♡ Flatpak

What is Flatpak

Flatpak is a new way of distributing applications.  Each application runs in its own isolated environment called sandbox with all its dependencies being provided as part of the Flatpak and with no access to other programs. This way every user runs the exact same application in the exact same environment no matter what Linux distribution they use.  The applications inside the sandbox are also limited to what system resources they can access, which provides greater security.

Kontact Stable on Flathub

The latest stable version of Kontact is now available on Flathub, the official app store for Flatpaks. So even if your distribution does not ship the latest version of KDE Applications or Kontact you can now easily install it from the Kontact Flatpak without having to worry about breaking your system.

How to get it? Follow this setup guide on Flathub to install Flatpak and to set up the Flathub repository.

Then you can install Kontact Flatpak from Discover or you can install it manually from terminal:

flatpak install flathub org.kde.kontact

You should see Kontact icon in your installed apps, or you can run it manually from terminal again:

flatpak run org.kde.kontact

Your feedback is welcomed, taming the entire Kontact to run inside of a tiny sandbox wasn’t an easy task and we may have missed something. You can report issues in the Kontact Flathub repository, reach us on the KDE PIM mailing list or via IRC on the #kontact channel on Freenode.

Kontact Nightly

We also have nightly builds of the Kontact development branch. You can use the development Kontact Flatpak to see if your bug has been fixed, to help us with testing Kontact and to provide early feedback on new features before they are released. The nightly Kontact Flatpak runs completely isolated from the stable Flatpak as well as from system-wide installation, which means that your data and configuration are completely safe. We hope that offering this easy and safe way of running the latest development builds of Kontact to a wide audience we will receive more feedback and early testing, leading to better and more polished releases in the future.

To install the nightly build you first need to add the kdeapps-testing Flatpak repository:

flatpak remote-add --if-not-exists kdeapps-testing --from https://distribute.kde.org/flatpak-apps-testing/kdeapps.flatpakrepo

And install Kontact Nightly:

flatpak install kdeapps-testing org.kde.kontact//master

To run the nightly version, use

flatpak run org.kde.kontact//master

You can read a more detailed guide to Flatpak on our community wiki.

Remember to run flatpak update</code>` from time to time to always get the latest version.

Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…