March and April in KDE PIM

Following the post about what happened in KDE PIM in January and February let’s look into what the KDE PIM community has been up to in March and April. In total 38 contributors have made almost 1700 changes. Big thanks to everyone who helped us make Kontact better!

April 2020 Release

A new bundle of KDE applications has been released in April, including Kontact with its many bugfixes and improvements.

KDE PIM Sprint

Every year in April the PIM team meets in Toulouse in France for a weekend of discussions and hacking. This year due to the coronavirus it wasn’t possible for us to meet so instead we held a virtual KDE PIM Sprint. You can read the sprint agenda as well as Volker’s report from the sprint.

To highlight some of the topics discussed

  • Moving KDAV into KDE Frameworks
  • KParts usage in Kontact
  • Remove the Kolab Resource, encourage users to use IMAP+DAV instead
  • Data provider plugins for the KCalendarCore library
  • Porting Account Wizard away from Kross (port to QML, eventually replace it completely by KAccounts)

KMail

Bugfixes

KMail has received its usual dose of bugfixes, mostly those:

  • Fixed a crash when sending an email (Laurent Montel, bug 419473)
  • Fixed a crash when adding too many recipients (Jonathan Marten, D28876)
  • Fixed a bug when KMail showed only a part of an HTML email (Laurent Montel, bug 419949)
  • Fixed a bug when KMail did not correctly display email answer in HTML mode (Laurent Montel, bug 418482)
  • Fixed broken message status filter (Laurent Montel, bug 419720)
  • Fixed the maildir backend creating junk folders around and not storing the path configuration properly (Igor Poboiko, D27722, D27906 and D28634)
  • Fixed a crash when configuring the POP3 resource (Daniel Vrátil, bug 419726)
  • Fixed name of the top-level folder created by the EWS resource (Igor Poboiko, bug 416663)
  • Fixed the EWS resource not storing user password properly (Igor Poboiko, bug 414789)

Improvements

There were some exciting improvements to KMail as well: Sandro Knauß has implemented support for Protected Headers for Cryptographic E-Mails. This means that we also send a signed/encrypted copy of headers and display the signed/encrypted headers if available and ignores the unsecure headers. Currently we don’t obfuscate the subject to not break current workflows. Those things will be improved later on. Sandro together with Thomas Pfeiffer get a funding from nlnet to improve mail encryption. That means there will be more improvement happen the next months. The next topic they will look at is to add Autocrypt support for KMail.

Volker has improved the look and feel of the “HTML Content” and “External References” warnings in emails.

Improved look and feel of the "HTML Content" and "External References" warnings

As the Libre Avatar service has come back from the dead a while ago, so did now the support for it in KMail. The ‘Export to PDF’ feature which we introduced in the previous report has been polished (Daniel Vrátil, D27793).

The ‘Move To Trash’ code has been optimized so that deleting large amounts of emails should now be faster.

For developers it is now possible to open Chromium DevTools inside the message viewer pane to make it easier to debug message templates.

KOrganizer

Bugfixes

  • Fixed crashes in category and filter managers (Allen Winter)
  • Fixed bug when single instance of a recurring event couldn’t be changed (Shashwat Jolly, bug 410758)
  • Fixed crash when creating a new Todo from Kontact (David Faure, bug 420192)
  • Fixed ‘Only resources can modify remote identifiers’ error when re-editing event (Igor Poboiko, bug 407965)
  • Fixed the DAV resource getting stuck when parse error occurs (David Faure, D27858)

Improvements

The Google Calendar and Google Contacts backends have been merged into a single Google Groupware resource (Igor Poboiko, D28560). The change should be mostly transparent to users, the old backends will be migrated to the new unified backend automatically after update. During this Igor also fixed various bugs and issues in the backends and the LibKGAPI library, big kudos to him!

The DAV resource is now able to synchronize the calendar color from KOrganizer to the DAV server (David Faure, D28938). Related to that, the menu to configure calendar color in KOrganizer has been simplified by removing the “Disable Color” action.

It is now easier to recognize and set the default calendar and the event editor now respects the settings correctly.

It is now easier to see the default calendar in KOrganizer

KJots

KJots, the note taking application, which has been on life support for 5 years, has received some love recently thanks to Igor Poboiko. Most of the things were happening under the hood: some ancient dusty code has been dropped, some refactoring happening, etc. However, if you still use KJots, you might also notice quite a number of changes too. And if you don’t, it’s a good time to consider using it :)

Bugfixes

  • Fixed a data loss issue due to bugs in the Akonadi Maildir resource, which is used as a KJots backend
  • Fixed a crash on startup (bug 420228)
  • Fixed multiple actions for the same shortcut (bug 384000)
  • Fixed bookmarks support (D29073)
  • Fixed export to plain text and to HTML
  • Fixed random scrollback jumps (bug 195828)
  • Fixed nested bullet lists breaking the undo stack (bug 256001)

Improvements

  • Link destination is displayed in tooltip (D29289)
  • Ctrl+click follows the link (bug 244846)
  • Priting support has been revived (D29045)
  • The text editing widget now supports different headings (bug 230317)
  • Improved support for nested bullet lists

Future

Igor has quite huge plans for the future of KJots. First of all, more bug squashing. Secondly: ability to store notes in Markdown format, synchronization with online services (thoughts are on OwnCloud/NextCloud, or proprietary Evernote). On a lesser scale, the port to the same text editing component as used by KMail email composer is being considered, which will give KJots more text-editing features. There are also plans to add a support for inline checkboxes introduced in Qt 5.14, which would allow making checklists and TODO-lists in KJots, and ability to sort books and pages by their modification date (so more relevant would pop up first).

Other components

Other parts of PIM has also received bugfixes and improvements. Kleopatra, the certificate management software, now displays GPG configuration tabs and option groups always in the same order (Andrey Legayev, T6446). A bug in Akregator has been fixed that could have caused some feeds to have an icon missing (David Faure, D28581). KAlarm has received a bunch of UI improvements as well as some smaller features - for instance it is now possible to import alarms from multiple calendars at once and the calendar list is now sorted by name (all by David Jarvie).

Common Infrastructure

Lots of work went into modernizing Akonadi, the “backend” service for Kontact. One major change was switch to C++17 and some initial usage of C++17 features internally (public API is still C++11-compatible). Widgets for managing Tags have been improved and polished and the deprecated ItemModel and CollectionModel have been removed.

The KIMAP library has been optimized to better handle large message sets (Daniel Vrátil, D28944). The KLDAP library can now connect to LDAP server using SSL encryption (Tobias Junghans, D28915), alongside the existing TLS support.

Volker Krause has been working on preparing the KDAV library (which implements the DAV protocol) into KDE Frameworks.

Laurent Montel has been working throughout the entire PIM codebase, preparing it to port to Qt6, once it’s available.


Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

January and February in KDE PIM

Following the post about what happened in KDE PIM in November and December by Volker, let’s look into what the KDE PIM community has been up to in the first two months of the new year. In total 23 contributors have made 740 changes.

KMail

  • A warning is shown when message composer is opened by clicking on a URL that asks for a file to be attached

Files attached by clicking on a mailto URL

  • DKIM validation support is enabled by default now

DKIM validation information

  • The email can now be easily exported to PDF
  • Improved support for rendering markdown in emails
  • “Important email” icon is now used consistently in KMail
  • Fixed “Move Message to Folder” dialog not accepting certain characters (#415850)
  • Fixed large emails not being displayed (#387061)
  • Fixed attachments being lost when message is saved as draft (#416369)

KOrganizer

  • Fixed restoring last used calendar in incidence editor (#411191)
  • Fixed fallback to the default calendar in incidence editor
  • Fixed a crash in release builds (#417794)

Akregator

  • Fixed filename extensions being translated (#416983)

KAlarm

  • Fixed message captions
  • Added warning when no default alarm calendar is set during archiving
  • Fixed crashes
  • Better handling when calendars are added or removed

Common Infrastructure

Laurent has been working on porting our code away from API that has been deprecated in Qt 5.15. Volker has been working on removing KDBusConnectionPool from all of KDE PIM.

For network communication we now use some safer defaults - for example we enabled HSTS by default and we don’t allow redirects to less-safe protocols (i.e. from https:// to http://).

KMail and some other components have begun integrating KUserFeedback to provide some basic telemetry information about usage.


Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

Q_PRIVATE_SLOT with new connect syntax

When using PIMPL, we sometimes want to move implementation of slots into the private class as well. In order for Qt to be able to invoke those slots that formally exist only in the private class (which usually is not a QObject), we use the Q_PRIVATE_SLOT macro in the main class. It allows Qt to invoke the slot method, even though it exists in the private class.

Let’s have a short example:

/// mybutton.h

class MyButtonPrivate;
class MyButton : public QPushButton {
    Q_OBJECT
public:
    explicit MyButton(QWidget *parent);
    ~MyButton() noexcept override;

private:
    std::unique_ptr<MyButtonPrivate> const d_ptr;
    Q_DECLARE_PRIVATE(MyButton);

    Q_PRIVATE_SLOT(d_func(), void onClicked(bool));
};

/// mybutton.cpp

class MyButtonPrivate
{
public:
    void onClicked(bool clicked) {
        qDebug() << "Clicked!";
    }
};

MyButton::MyButton(QWidget *parent)
    : QPushButton(parent)
    , d_ptr(std::make_unique<MyButtonPrivate>())
{
    // Connecting to slot on "this" (MyButton*), although "onClicked" is defined in MyButtonPrivate
    connect(this, SIGNAL(clicked(bool)),
            this, SLOT(onClicked(bool)));
}

MyButton::~MyButton() noexcept = default;

Q_PRIVATE_SLOT does not create any new method in the MyButton class. The way Q_PRIVATE_SLOT works is that it just instructs moc to generate a metacall that looks like obj->d_func()->onClicked(val) instead of obj->onClicked(val), like it does for normal slots or invokables.

This approach has one big disadvantage: it means that Q_PRIVATE_SLOTs can only be invoked through the old QObject::connect() syntax.

So far I’ve been using a simple workaround to get all the compile-time checks that I would get with the new connect syntax normally:

connect(this, &MyButton::clicked,
        this, [this](bool clicked) { d_func()->onClicked(clicked); });

Here we use a lambda to forward the call to the actual PIMPL’ed slot. This is somewhat better than the old connect syntax but IMO it has two major drawbacks:

It’s hard to read - it’s difficult to immediatelly decipher what method the call is actually being forwarded to.

It’s tedious to write - it’s a lot of boilerplate code to be written and if there are too many arguments it becomes quite ugly. C++14 generic lambdas improve the situation a bit since we can use auto instead of spelling out the argument types, but I don’t think it makes the code necessarily better to read:

connect(this, &MyObject::mySignal,
        this, [this](const auto &foo, auto bar, auto *baz) {
            d_func()->mySlot(foo, bar, baz);
        });

It got me thinking if there might be some way to auto-generate the forwarding lambda and be able to just use the pointer-to-function here somehow.

In the end I came up with this tiny template function, which takes the d-pointer and the pointer to the PIMPL’ed slot and returns a generic lambda which gets passed into QObject::connect and which forwards arguments to the real slot method.

template<typename DPtr, typename Slot>
auto priv_slot(DPtr &&dptr, Slot &&slot)
{
    return [&dptr, &slot](auto && ... args)
    {
        (dptr->*slot)(std::forward<decltype(args)>(args) ...);
    }
}

The result has all the benefits of the new QObject::connect() syntax without the ugliness of the “forwarding lambda”:

connect(this, &MyObject::mySignal,
        this, priv_slot(d_func(), &MyObjectPrivate::onSignal));

I'm Going to Akademy!

I'm Going to Akademy!

In just five days I’ll be on my way to Akademy! I’m so excited to meet with all my friends from KDE! After missing the conference weekends in Almería and Vienna, I’ll be able to get the full Akademy experience again - including delivering a talk!

Build Expressive APIs with Modern C++

I’ll be giving a talk about how to use some cool features from C++17 (even if you cannot use C++17!) in your code to make it easier for others (and yourself) to understand the intentions of the code, which helps improve productivity and reduce bugs and errors. The talk will be on Sunday at 14:35 in room U4-08.

KDE PIM BoF

The KDE PIM team will have a BoF session on Monday morning (10:30 - 12:30) in room U1-04. If you want to talk to us about anything KDE PIM related, feel free to stop by!

Other than that my main intention is to make use of the whole week to do some intensive hacking on Akonadi, in-person debugging and fixing bugs :)

See you all in Milan!

Kontact and Google Integration Issues

Lately there were some issues with the Google integration in Kontact which caused that it is no longer possible to add new Google Calendar or Gmail account in Kontact because the log in process will fail. This is due to an oversight on our side which lead to Google blocking Kontact as it did not comply with Google’s policies. We are working on resolving the situation, but it will take a little bit.

Existing users should not be affected by this - if you already had Google Calendar or Gmail set up in Kontact, the sync should continue to work. It is only new accounts that cannot be created.

In case of Gmail the problem can mostly be worked around when setting up the IMAP account in KMail by selecting PLAIN authentication1 method in the Advanced tab and using your email and password. You may need to enable Less Secure Applications in your Google account settings in order to be able to log in with regular email address and password.

If you are interested in the technical background of this issue, the problem comes from Google’s OAuth App Verification process. When a developer wants to connect their app to a Google service they have to select which particular services their app needs access to, and sometimes even which data within each service they want to access. Google will then verify that the app is not trying to access any other data or that it is not misusing the data it has access to - this serves to protect Google users as they might sometimes approve apps that will access their calendars or emails with malicious intent without them realizing that.

When I registered Kontact I forgot to list some of the data points that Kontact needs access to. Google has noticed this after a while and asked us to clarify the missing bits. Unfortunately I wasn’t able to react within the given time limit and so Google has preemptively blocked login for all new users.

I’m working on clarifying the missing bits and having Google review the new information, so hopefuly the Google login should start working again soon.

  1. Despite its name, the PLAIN authentication method does not weaken the security. Your email and password are still sent safely encrypted over the internet. 

March and April in KDE PIM

The KDE PIM community has been working hard in March and April to fix various bugs and bring new features and improvements to Kontact, Itinerary and other KDE PIM projects. Here is a summary of some of the changes that landed in the past two months.

KDE PIM Sprint

The team has met for a weekend for a traditional spring PIM sprint in Toulouse at the beginning of April. We focused on solving some long-standing issues in Akonadi, prepared some of the KDE PIM libraries to be moved to KDE Frameworks and discussed outreach and how to attract more people to KDE PIM.

You can read David’s report from the sprint.


KMail

KMail has received a nice batch of bugfixes for the KDE Applications 19.04 release as well as some small features, which will be available in the 19.08 release.

Improvements and New Features

The email composer has received improvements to support UTF-8 emojis (Laurent Montel, KDE Applications 19.08) and to support composing rich-text emails using Markdown with syntax highlighting (Laurent Montel, KDE Applications 19.08).

Additionally, notification when sending email fails is now less intrusive (Laurent Montel, KDE Applications 19.08).

Bugfixes

  • Fixed crash on shutdown due to mishandling of QWebEngineProfile (Jonathan Marten, D19559)
  • Fixed reconnecting to IMAP server when connection is lost during login (David Faure, D20028)
  • Fixed identity not appearing after creation (Laurent Montel, bug 391631)
  • Fixed deleting spam messages (Laurent Montel, bug 406324)
  • Fixed custom header search query failing with inconclusive error (Laurent Montel, bug 405456)
  • Fixed trying to send a message without any recipients fails quitely (Laurent Montel, bug 402577)
  • Fixed a crash switching on an ad-blocked (Laurent Montel, bug 405434)
  • Fixed a crash when opening a vCard attachment (Laurent Montel, bug 405791)

KOrganizer

During the PIM sprint, Volker Krause did a major work on cleaning up KCalCore, a library that implements the iCal standard to store events and tasks information, so that the library can be moved to KDE Frameworks. You can read more about Volker’s effort in his blog post.

New Features and Improvements

It is now possible to move event from one calendar to another in KOrganizer by changing the calendar in the incidence editor (David Faure, KDE Applications 19.08).


KAddressBook

Volker also worked on preparing the KContacts library, a vCard standard implementation, to be moved to KDE Frameworks alongside KCalCore. You can read about this on his blog as well.

Improvements

As part of his code cleanup, Volker has removed the DataMatrix from contact display, so we only display the QR code now (Volker Krause, KDE Applications 19.08), and did massive clean up of the contact display code. When displaying contact address on a map, KAddressBook now defaults to OpenStreetMaps (Volker Krause, KDE Applications 19.08).

Thanks to Laurent Montel it is now also possible to send SMS messages from KAddressBook through KDE Connect (KDE Applications 19.08).

Bugfixes

  • Fixed dialing contact’s phone-number from KAddressBook through KDE Connect (Volker Krause, D20353)

Akregator

Bugfixes

  • Make the feed URI the baseUrl for article previews (Pierre Ducroquet, D19739)
  • Fixed Delete key not working after using search (Laurent Montel, bug 394946)

Common Infrastructure

Improvements

David Faure and I have spent large part of the PIM sprint investigating and digging into two major issues we currently have in Akonadi: database deadlocks and a bug known as “multiple merge candidates”. We were unable to determine the real cause for the “multiple merge candidates” bug, so it remains unfixed for now. As a result of digging through the code base, however, David has produced a set of patches to hugely improve handling of database deadlocks and transaction rollbacks in the Akonadi server. He also did improve the sync scheduling code. I have removed a large chunk of code by removing mostly unused code to handle Collection references and LDAP/Kolab contact merging.

New Features

  • LibKGAPI: support for Team Drives API for Google Drive (David Barchiesi, T10521)
  • KItinerary: support for parsing train booking from SNCF confirmation emails (Volker Krause, bug 404451)

Bugfixes

  • Akonadi: fixed sync getting stuck after failure (David Faure, D19487, bug 399167)
  • Akonadi: fixed race conditions in Attribute handling (David Faure, D19556, D19632)
  • Akonadi: fixed crash when in resources when handling a collection change (Daniel Vrátil, bug 403642)
  • Akonadi: fixed crash when an Akonadi client unexpectedly disconnects from the server (Filipe Azevedo, D19983)

Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

Unified Mailboxes in KMail

Today KMail has gained a new cool feature that has been repeatedly requested in the User survey last year as well as on forums and social networks: Unified mailboxes.

Unified mailboxes offer not only a unified inbox - a single "Inbox" folder showing emails from inboxes of all your accounts, it also provides unified sent and drafts folders by default. But we did not stop there: you can create completely custom unified mailboxes consisting of any folders you choose. You can even customize the default ones (for example exclude an Inbox from a particular account).

Some obligatory screenshots:

Unified Mailboxes in KMail Unified Mailboxes configuration

The feature will be present in the December release of KDE Applications.

 

Do you want to help us bring more cool features like this to Kontact?

Then take a look at some of the junior jobs that we have! They are simple mostly programming tasks that don't require any knowledge of Akonadi or all the complexities of Kontact. Feel free to pick any task from the list and reach out to us! We'll be happy to guide you. Read more here...

KDE PIM Junior Jobs are opened!

Do you want to help us improve your favorite PIM suite but you were always scared by its size and complexity? Well, fear no more! We have collected a bunch of simple and isolated tasks in various parts of the PIM suite that require none or just very basic understanding of how the entire Kontact and Akonadi machinery works. We have documented them and we are prepared to guide you and help you to accomplish the tasks. Those are small simple tasks, but they will make many users (and PIM developers) very very happy.

I'm in! What do I need to know?

Just some C++ and maybe a bit of QML if you want to go for the QML tasks. We don't expect you to know anything about Akonadi or how the entire PIM thing works as most of the tasks are pretty self-contained (although you can read up on the basic concepts and architecture if you are interested).

Cool, what do you offer?

We have tasks to improve the look of KAddressbook's contact list, contact view, and contact editor. If you prefer working on KOrganizer, you can help us to make the event view look more modern. We would also like to improve the Account Wizard experience by porting it to QML and improving Gmail/Google Calendar and Contacts integration. Of course, the key part of Kontact is KMail and even there we have a few places that can be improved: we would like to improve the IMAP quota warning and add support for Autocrypt. And finally, you can also make life easier for other KDE PIM developers by improving our debugging tool, the Akonadi Console: we want to be able to save the output into JSON and load it again, alphabetically sort some of the lists, make working with the DB console a bit more comfortable and be able to restart Akonadi agents whenever we want to.

There's also a very cool effort ongoing to allow integration between Kontact and MyCroft, the opensource voice assistant. For this, we need help improving a command line tool that's used as a bridge between MyCroft and Kontact.

If you don't know any programming but you would still like to help, we have some non-programming tasks as well! Sure! We are working on a new website for Kontact and we could use help with both design and writing content for it! We also need help improving our user documentation, cleaning and updating our wikis on community.kde.org and userbase.kde.org or cleaning up our bug tracker. If you want to help with any of that, get in touch with us on the kde-pim mailing list!

You can find the full list of junior jobs on Phabricator.

Haven't found anything interesting? Don't worry, we will keep adding more over the time, so just check the list every now and then. Or do you have your own idea how to improve KDE PIM and you just don't know where to begin? Get in touch with us and we will help you!

Now how do I get started?

1) Get in touch with us

To make sure several people won't try to solve the same thing, it is the best to get in touch with the PIM community first so we can look at the single topics in more details. Some of the descriptions of the tasks are intentionally a bit vague as there are multiple ways how to approach or solve the problem. It's always better to talk about the options first so that no time is wasted on approaches that won't work.

2) Get your development environment set up

The KDE PIM community wiki contains articles on how to develop KDE PIM inside a Docker container. Alternatively, as most of the changes are pretty isolated, you should be able to compile just a single component from source against your distribution packages (you will just need to install some development packages first).

3) Pick a task

Pick one of the tasks linked above, or just look at all the junior job in Phabricator. They span different topics, different components and are of different complexity and size. If you find a particular task that you would like to work on, assign it to yourself and get working! If someone else already has the task assigned, you can ask if they maybe want some help, or just look for another task.

4) Get to work!

Fire up your favorite IDE and start working! If you need any help with the task - from finding the right repository and code, through getting the program compiled to being stuck on a bug or something not working - just ask us! You can ask in the Phabricator task or send an email to the kde-pim mailing list and some of the PIM devs will help you.

Also, don't feel limited by the description of the tasks - feel free to do only part of the task, or do even more than what's in the task description. If you think you have a better idea how to solve something, let us know in the Phabricator task.

I have the code, what's next?

Awesome! Now it's time to upload the code for review. You can use the arcanist command line tool, or you can just generate a diff and upload it manually via the web interface. Don't worry if you don't know whom to assign for review, Phabricator sends the notification the entire PIM team automatically.