KMail: “Multiple Merge Candidates” error and how to fix it

Standard

If you can’t synchronize a folder in KMail and you are seeing “Multiple Merge Candidates” error after the synchronization fails, here’s a how to fix the folder to make it synchronize again – basically you force KMail to forget and re-sync the folder again.

 

  1. Open Akonadi Console.
  2. Go to the Browser tab.
  3. Right-click the broken folder and select “Clear Akonadi Cache” – this will remove all emails from the folder in Akonadi. This will NOT delete your emails on the server.
  4. Akonadi Console will freeze for a while, wait until it unfreezes (sorry, it’s just a developer tool, we don’t have a very good UX there :-)).
  5. Logout and login to make sure all PIM components are restarted.

 

After login start KMail (or Kontact) and hit “Check mail“. KMail will now re-download all emails from the previously broken folder. This may take a while depending on how large the folder is and how fast your internet connection is. After that the synchronization should work as expected.

 

In the upcoming KDE Applications 16.12.1 release Akonadi will have a fix that fixes the reason why the “Multiple Merge Candidates” error occurs, so hopefully in the future you should not see this error anymore.

Akademy 2016 is over :(

Standard

It’s actually been over for two days, but I’m still sitting in Berlin and only now got to write something.

As every year, it was great to see all my friends and fellow hackers again. Thanks everyone for being so awesome, I enjoyed every day of QtCon and Akademy with you. Can’t wait to meet everyone again next year :-)

In the terms of KDE PIM, this year’s Akademy was very productive. We had our KDE PIM BoF session on Monday afternoon, where we spent most of the time discussing KDE PIM User Survey – a plan of mine to get more information about our users and their use cases. The results will help us, the KDE PIM devs, to better understand how our users use our software and thus prioritize our focus. We ended up with an initial set of questions we intend to ask our users and next week I’ll meet with some more KDE PIM hackers that could not attend Akademy and we will finalize the set of questions so that we can publish the survey later this month.

We also talked about some other topics on the meeting, like releasing of some of our libraries that Kube wants to use and so on.

You can read the mostly complete meeting notes on the KDE PIM wiki.

Outside of the BoF session we touched the topic of KDE PIM sprints and meetings. We want them to be more focused in the future, i.e. having a specific topic for each meeting that we will all work on together. We hope to do one meeting in Autumn this year to finish porting KCalCore away from KDateTime and KDELibs4Support, then a Spring meeting in Toulouse (which has become our new regular place for Spring sprints), then Randaaaaaaaaaaa (which gives us full 6 days of uninterrupted hacking with only small breaks to eat Mario’s chocolate :-)) and then it’s Akademy time again!

Oh and I can’t forget to mention that the KDE PIM team was awarded the Akademy Award for our work on, well, KDE PIM :-). It was a great feeling to stand on the stage knowing that people appreciate our work.

Regarding my PIM work during Akademy, I think this year was pretty good. I did my share of partying during QtCon, so I could spent most of Akademy days hacking from morning until they kicked us out from the venue, and then continuing with some more hacking in the KDAB office until late night. Already before the event I merged a big change that improves the Akonadi change notification system. I managed to polish it during Akademy and fix several crashes and bugs.

Another big change was to our test-suite. It contains among other things integration tests, where we run an actual Akonadi server in an isolated environment (so that it does not touch any real data) and test whether clients interact with it as they are supposed to do. For these integration tests we’ve been only using the SQLite database until now, but I have now enabled MySQL and PostgreSQL too, so we run each test three times – once for each of the backends. This has revealed several corner-case issues that we weren’t aware of until now. The test still run into some issues on the CI on build.kde.org but locally they pass for me (with only one exception). Addressing those issues is on the top of my todo list now.

I also started working on an experimental XML->C++ generator, which would allow me to get rid of some 12,000 lines of hand-written C++ code that implements the communication protocol between Akonadi server and the clients. Instead I will generate the code from a simple XML. So far I managed to get it to generate a code that compiles, but there’s still a lot of work ahead to make it generate an optimal and correct code.

I’ll spend the next week meeting all my colleagues from KDAB, which I’m really looking forward to. Although I know many of them from KDE, there are lots of people I haven’t met yet, so it will be great to attach faces to the names. After that, it’s back to Prague and to regular work (and some more Akonadi hacking ;-)).

Oh and if you haven’t heard yet, KDE is celebrating 20th birthday. Go check out the timeline of KDE and get the amazing “20 Years of KDE” book!

Akonadi – still alive and rocking

Standard

It’s been a while since I wrote anything about Akonadi but that does not mean I was slacking all the time ;) The KDE PIM team has ported PIM to KDE Frameworks 5 and Qt 5 and released the first KF5-based version in August 2015 and even before that we already did some major changes under the hood that were not possible in the KDE4 version due to API and ABI freezes of kdepimlibs. The KF5-based version of Akonadi libraries (and all the other KDE PIM libraries for that matter) have no guarantees of stable API yet, so we can bend and twist the libraries to our needs to improve stability and performance. Here’s an overview of what has happened (mostly in Akonadi) since we started porting to KDE Frameworks 5. It is slightly more technical than I originally intended to, sorry about that.

Human-readable formats are overrated

As you probably know Akonadi has two parts: the Server (that manages the data and resources) and client libraries (that applications use to access the data managed by the server). The libraries need to talk to the Server somehow. In KDE4 we were using a text-based protocol very similar to IMAP (it started as RFC-compliant IMAP implementation, but over the time we diverged a bit). The problem with text-based protocol and large amount of data is that serializing everything into string representation and then deserializing it again on the other end is not very effective. The performance penalty is negligible when talking to IMAP servers over network because the network latency hides it. It however shows when everything is happening locally. So we switched from a text-based protocol to a binary protocol. That means we take the actual representation of the data in the memory (bit by bit) and write it to the socket. The other side then just takes the binary data and directly interprets them as values (numbers or strings or whatever). This means we spent almost zero time on serialization and we are able to transmit large chunks of data between the server and the applications very, very efficiently.

Let’s abuse the new cool stuff we have for things we did not originally designed it for

The communication between clients and server needs to work in two directions. It’s not just the clients sending requests to server (and server sending back replice), we also need a mechanism for the server to notify the clients that something has changed (new event in a calendar, email marked as read, etc.). For this we were using DBus signals. The clients could connect to a DBus signal provided by the Akonadi Server and when something changed, the server notified the clients via the signal. However during initial synchronization or during intensive mail checks the amount of the messages sent over DBus by Akonadi was just too high. We were clogging the DBus daemon and the transmission of messages via DBus is not cheap either. But hey, we have an awesome and super-fast binary protocol, why not use that? And so we switched from using DBus for change notifications to sending those notifications through the same mechanism that we use for all other communication with the server. In the future it will also allow us tu even more customize the content of the notification thus further improving performance.

Pfff, who needs database indexes?

We do! Once we switched to the binary protocol we found that we are no longer waiting for the data from database to be serialized and sent over to client, but that we are waiting for the database itself! I sat down and look at EXPLAIN ANALYZE results of our biggest queries. Turns out we were doing some unnecessary JOINs (usually to get data that we already had in in-memory cache inside the Server) that we could get rid of. SQL planners and optimizers are extremely efficient nowadays, but JOINing large tables still takes time, so getting rid of those JOINs made the queries up to twice faster.

One unexpected issue I found was that the database was spending large amount of time on “ORDER BY … DESC” on our main table (yes, we query results in descending order – this way we can show the newest (= usually most relevant) emails in KMail first, while still retrieving the rest). PostgreSQL users will be happy to know that adding a special descending index sped up the queries massively. MySQL users are out of luck – although MySQL allows to create a descending index on a column, it does not really do anything about it.

Splitting libraries is too mainstream, we merge stuff!

One of the things that I’ve been looking forward to for a very long time was making the Akonadi server a private part (an implementation detail) of the Akonadi client libraries. In KDE4 versions we had to maintain a backwards compatibility of the Akonadi protocol (the text-based one I mentioned earlier) as it was considered a part of public API. This was extremely limiting and annoying for me as a maintainer, as it was making fixing certain bugs and adding new features unnecessarily hard. Historically Akonadi server was a standalone project and it was expected that 3rd party developers would write their own client libraries in their own toolkits/languages. Unfortunately that never happened and the KDE Akonadi client libraries were the only client libraries out there that were actively developed and used (there were some proof-of-concept GLib/Gtk client libraries, but never used seriously).

So, since KDE Applications 15.08 the Akonadi server has no public API and writing custom client libraries is not officially supported. The only official way to talk to the server is through the KDE Akonadi client libraries, which is now the only public API for Akonadi. This may sound like a bad decision, like closing ourselves down from the world, like if we don’t care about anyone else but KDE and Qt. And it’s sort of true – we were waiting for almost a decade for someone else to start writing their client libraries, but nobody did. So why bother? On the other hand the only actual and real user of Akonadi – KDE – benefits much more now – for example the binary protocol is optimized so that (de)serializing Qt types (like QString or QDateTime) is very efficient because we can use the format that Qt uses internally. If we were to be “toolkit agnostic”, we would have to waste time on converting the data to some more standard representation and nobody would win.

To finally get to the point: today I took the Akonadi client libraries (that lived in kdepimlibs.git) and merged them to akonadi.git repository, where the Akonadi server is – at least locally on my machine, still need to fix some build issues before actually pushing this, but I expect to do it tomorrow. In other words the entire Akonadi Framework now lives in a single self-contained git repository. This brings even more benefits, mostly from maintainer point of view. For instance we can now share more code between the server and the libraries that we previously had to duplicate or expose via some private shared library.

The kdepimlibs.git will still contain some libraries for now that we yet have to figure out what to do with, but I guess that eventually kdepimlibs.git will meet the same fate as kdelibs.git – being locked down and preserved only for historical reference.

The Cheese Dependency

In September last year the KDE PIM team also met in Randa in Swiss Alps. I was totally going to blog about it, but then other things got into way and I kept delaying it further and further until now. So with an awkward 5 months delay: huge thanks and hugs to the entire Randa meetings staff and one more hug to Mario just for being Mario. In Randa we met to discuss where KDE PIM should go next and how to get there. After several days on intensive talking we outlined the path into future – you probably read about it already in some of the blogs about AkonadiNext from Christian and Aaron, so I won’t go much into that.

To list some of the visible and practical results – we now use Phabricator to coordinate the work in the PIM team and to better communicate what is happening and who’s working on what. There’s a nice backlog of tasks waiting to be done, so if you want to help us make PIM better feel free to pick up some task and get to work! Furthermore we looked into cleaning up some of the old code and optimizing some critical code-paths – basically a continuation of an effort that started already during Akademy in A Coruña. Some of the changes were already implemented, some are still pending.

Lord of the PIM: The Return of The KJots

One of the major complaints we heard about the new KF5-based KDE PIM was the disappearance of KJots, our note-taking app. Earlier last year, on a PIM sprint in Toulouse, we decided that we need to reduce the size of the code base to keep it maintainable given the current manpower (or rather lack thereof). KJots was one of the projects we decided to kill. What we did not realize back then was that we will effectively prevent people from accessing their notes, since we don’t have any other app for that! I apologize for that to all our users, and to restore the balance in the Force I decided to bring KJots back. Not as a part of the main KDE PIM suite but as a standalone app. I have yet to make a first release that packagers can package, but it already builds and is reasonably usable. I’m not planning on developing the application very actively – I’ll keep it breathing, but that’s about it. That’s all I can afford. If there’s anyone who would be interesting in maintaining the application and developing it further (it’s a rather small and simple application), feel free to step up! When the app reaches certain quality level, we can start thinking about merging it back to KDE PIM.

Is that all?

Yes. No. Well, it’s all for today. There is much much more happening in KDE PIM – Laurent did tons of work on of refactoring and splitting the monolithic kdepim.git repository into smaller, better reusable pieces and now seems to be messing around Akregator, and Sandro is actively working on refactoring the email rendering code and calendaring. But I’ll leave it up to them to report on their work :) And of course there’s much more planned for the future (as always), but this blog post already got a bit out of hand, I’ll report on the rest maybe next time I “accidentally” have an energy drink at 11 PM.

And as always: we need help. Like, lots of it. KDE PIM might look huge and scary and hard to work on, but in fact it’s all rainbows and unicorns. Hacking on PIM is fun (and we are fun too sometimes!), so if you like KDE (PIM) and would like to help us, let’s talk!

No Gmail integration in 4.14 after all :(

Standard

Hi folks,

I’m sorry to bring bad news, but after trying to fight some last minute bugs in the new Gmail resource today, I realized that pushing the resource into KDE Applications 4.14 was too hurried, and so I decided not to ship it in KDE Applications 4.14. I know many of you are really excited about the Gmail integration, but there are far too many issues that cannot be solved this late in 4.14 cycle. And since this will probably be the last 4.x release, shipping something that does not perform as expected and cannot be fixed properly would only be disappointing and discouraging to users. In my original post I explained that I was working on the Gmail integration to provide user experience as close as possible to native Gmail web interface so that people are not tempted to switch away from KMail to Gmail. But with the current state of the resource, the effect would be exactly the opposite. And if the resource cannot fulfil it’s purpose, then there’s no point in offering it to users.

Instead I will focus on implementing the new native Gmail API and merging together the existing Google resources to create a single groupware solution that will provide integration will all Google’s PIM services – contacts, calendars, tasks and emails.

 

Improved Gmail integration in KDE PIM 4.14

Standard

Update: no Gmail integration after all, feel free to contact me if you want to help

Hi all,

I already teased publicly about the new Gmail resource on Google+ yesterday, now it’s time for some more explanations and…screenshots!

What is this about?

A native Gmail Resource for Akonadi that will bring much better integration of Gmail features into Kmail.

I’ve been talking about it and promising it for quite some time. But now thanks to some changes to the regular IMAP Resource that Christian Mollekopf has done recently, I could finally start working on it!

But…why?

Those who use Gmail know that Gmail does some things differently than most normal mail services. The biggest difference is that there are not really any folders with emails. Instead there’s one folder with all your emails and then there are labels, that you can assign to emails and then you can just filter your emails by the labels. And you can assign multiple labels to one email.

Yeah, but why bother? It already works quite well with the normal IMAP resource, right?

Yes, Gmail is able to hide this specialities from regular email clients so that they can still work with Gmail like with any other generic mail server, but at the cost of losing some features.

More and more often you can hear today that desktop email clients are dead and the future is in webmail (and cloud). And for many users who only have one email account this is true – why having KMail and KOrganizer etc. running, when they can have all this and more opened constantly in a single tab in their web browser? And the truth is, that Gmail is simply the largest mail provider in the world today. So if we want to persuade all these users to keep using KMail, we need to provide a user experience that is as close as possible to the native web interface. And for that we need a native Akonadi Resource ;-)

(Note: I’d like to avoid flamewars about “desktop clients are not dead vs. are dead” – I believe they are not dead for people who use more than one email account. They will cling to desktop clients until the dawn of the Gods, and even longer, but for normal users with just one mail account, it might be just matter of years to leave desktop clients. But who knows. impossible to see the future is).

Ok, so what’s the difference between Gmail and IMAP Resources?

The Gmail Resource supports some Gmail-specific IMAP extensions. In other words, it can speak and understands Gmail’s IMAP dialect. This means that the Gmail resource can handle the Gmail specifics better than the generic IMAP Resource:

1) Flattened folder hierarchy. This is best shown on screenshots: on the left, there’s a folder hierarchy as shown in KMail when using the current IMAP resource, on the right there’s the same account but synced via the new Gmail resource.

Folder hierarchy synced via the IMAP resource
Folder hierarchy synced via native Gmail Resource

2) One email to rule them all! As explained above, one email on Gmail can have multiple labels. But when you sync the mail via normal IMAP, you get a copy of that email in each folder. That means, that marking the mail as read will only mark as read that copy, but not the other copies in other folders, simply because KMail does not know they are effectively the same mail.

The Gmail resource is aware of this, an it syncs all your emails into one hidden folder and then just links them to the actual folders representing Gmail labels that you can see in KMail, so when you mark an email as read in any folder, it will mark it as read in all folders it’s linked into. Awesome, right?

3) OAuth authentication. The regular IMAP resource only supports the regular username-password authentication (and GSSAPI), which means that your password is stored in the computer somewhere, and if you use 2-step verification, you need to generate an app-specific password.

The Gmail resource has support for Gmail’s OAuth2, so you only enter your password once into Google’s web login, and the resource will then use a special tokens issued by Gmail with limited life-span to authenticate all your requests.

Gmail Resource authenticationGmail Resource: 2-step authentization support

The authentication is actually powered by LibKGAPI, a Qt/KDE library that implements various Google APIs, so it has the same look and uses the same code as the Akonadi Resources for Google Calendars and Google Contacts.

(Funny story and a question: I actually had to write a custom plugin for Cyrus-SASL to support XOAUTH2 mechanism, as upstream does not support it. Does anyone know whether there’s an existing implementation somewhere on the interwebs that I could use instead my crappy plugin?)

4) Simpler configuration. This is not really that big, bacause you don’t open the dialog very often, but I really like the configuration dialog a lot: simple and without complex options like encryption, mechanisms… This is simply because Gmail supports only a specific set of IMAP features, so I could just remove lot’s of stuff from the configuration dialog making it thinner and much easier to understand (IMO).

Gmail Resource configuration dialog

5) Push-notifications for all folders. The IMAP Resource can only monitor one folder for changes (using IMAP IDLE), because of certain technical restrictions. The folder usually is the Inbox. So if you have server-side filtering, you will never get push-notifications about new emails arriving to your other folders.

The limitation for watching only one folder applies to the Gmail Resource too. However since we understand Gmail, we can watch the “All Mail” folder, instead of the Inbox, so this way we get push notifications about emails from absolutely all folders (except for Trash and Junk folders, but who cares about these). Thinking about it, I could even remove the “Check Interval” option from configuration now.

Uhm ok, so what’s the state of the resource?

Currently the resource is still in a branch, waiting for some more features to be finished and for Christian to approve some of my changes to the IMAP resource (I’ll bribe him with some beer during Akademy, if necessary ;-)), and some changes must be done in KMail to properly support copying and moving of the linked emails, but other than that, it already works quite nicely :-)

That’s about it, see you next time :-)

Email Threading in KMail: Your Help is Needed!

Standard

Hi KMail users,

we have a little favor to ask from you :-) On the KDE PIM Sprint we discussed how to improve email threading in KMail by using Akonadi to store the information, so that KMail does not have to compute it every time. This would make opening a folder almost instant, all threads would be reconstructed immediately and it would massively improve CPU and memory consumption (so it’s totally something worth helping us with ;-)) More details on what else we discussed and implemented will follow in another blog post tomorrow.

To implement the threading caching, we need to know, whether in these days it still makes sense to support threading by Subject. It’s used as a fallback when grouping by standardized email headers (In-Reply-To, References) are missing, which used to be a case with buggy email clients years ago, but hopefully is better now, so we could drop it, which would massively simplify the algorithms.

So we would like you to disable threading by Subject, observe how much it breaks your threading (and potentially your workflow) and report back to us. To disable it, go to View ->Message List ->Aggregation -> Configure. There go to Groups & Threading tab and in Threading combobox select Perfect and by Reference. If the combo boxes are disabled, you have to click Clone Aggregation to clone the default settings, and use the clone.

View->Message List->Aggregation->Configure...

View->Message List->Aggregation->Configure…

Aggregation Configuration

Aggregation Configuration

If removing threading by subject would break threading and workflow for too many users, we will keep the settings and we will try to figure out another way to do it.

So please configure your KMails, and let us know in comments below this post, on IRC, kde-pim mailing list or through any other communication means (just please try to avoid using smoke signals and pigeons ;-))

Thank you for your help!

Akonadi 1.10.3 – with PostgreSQL fix

Standard

Akonadi 1.10.3 has been released, fixing (among other things) support for PostgreSQL 9.2 with Qt 4.8.5.

I write a blog specially about this update, because the fix requires updating the Akonadi database schema. The update will be performed on next Akonadi start and it can take some time, especially on larger databases, so we kindly ask users for patience.

Update: if your distribution ships Qt 4.8.5 with the PSQL driver patch reverted, please make sure to remove the revert before pushing Akonadi 1.10.3 to repositories.

Users of other backends (MySQL or SQLite) will not be affected by this update.

Big thank you for investigating and fixing this problem to Cédric Villemain.

Full changelog:

  • Fix crash when there are no flags to update during flags change
  • Fix crash on Akonadi shutdown when using PostgreSQL
  • Fix notification to clients about database upgrade
  • Send dummy requests to MySQL from time to time to keep the connection alive
  • Bug #277839 – Fix problem with too long socket paths
  • Bug #323977 – Check minimum MySQL version at runtime
  • Bug #252120, Bug #322931 – Use text instead of bytea column type for QString in PostgreSQL

Thanks to Volker Krause, Christian Mollekopf and Cédric Villemain for their contributions to this release.

What’s new in the Akonadi World

Standard

Hi!

I arrived back home from Akademy just a day ago and I already miss it. I enjoyed every single moment of it and had lots and lots of fun. Thanks everyone for making this such an awesome event, and especially to the local team. They did an incredible job!

This blog however will not be about Akademy (I will write one maybe later), but about Akonadi, core component of our PIM suite. As you probably already read or heard, Volker Krause has handed over to me maintainership of Akonadi.  It means really lot to me and I’ll do my best to be at least as good maintainer as he was (if that’s even possible) and I would like to thank him for his outstanding job he did writing and maintaining Akonadi.

I really believe in Akonadi, I like design of the framework and admire all the work guys have done on it since the beginning, long before I even dreamed about becoming a KDE contributor. I also believe that having a powerful and well-working PIM suite is the key for success of KDE (not just) in the enterprise world. Akonadi is more powerful than most of the competition out there, we just now need to focus bit more on stability and performance. I think we are doing pretty fine with stability, so I want to focus mainly on the performance side of Akonadi. In this bit more technical blog post I want to write about what I did recently, what I’m doing know and what are (some of) my future plans with Akonadi. As usually, huge thank you to Volker for his ideas, suggestions and comments about my ideas. We had a great discussion during Akademy and we (theoretically) solved many problems and bottlenecks that were bothering Akonadi for a long time, but nobody had time to look into it.

Batch Notifications

I started working on batch notifications after the KDE PIM sprint in March, merged it in May (I think) and hopefully have resolved all regressions by now. Akonadi server uses notifications to inform all clients about changes, like new items, changed items, removed items, etc. Notification is a simple data structure that is transferred via D-Bus to all clients. Before this patch there was one notification per each item which means that marking 500 emails as read had generated 500 notifications that had to be transferred via D-Bus. With batch notifications the server can create a single notification that references all 500 items. This saves a lot of D-Bus traffic and allows faster processing on the client side. This feature will be available in KDE 4.11.

MTime-based item retrieval

This was written during Akademy and will allow Vishesh to improve start-up time of the Akonadi Nepomuk Feeder. Until now the feeder had to fetch all items from Akonadi and all items from Nepomuk on start just to compare whether everything is up-to-date. With mtime-based item retrieval the feeder can ask Akonadi to hand over only items that have been changed since some given timestamp. In most cases that means 0 items. You have to agree that fetching no or just a few items instead of all of them will give us a notable performance boost during start. Albert allowed us to backport this to 4.11, so you will get this improvement in 4.11 as well.

External payload files without path

Another thing that has been implemented during Akademy is aimed to save some of your precious disk space. Because Akonadi is a cache for your data, it means that it has to store all your emails, contacts, events etc. somewhere. Smaller records are stored directly in the database. Larger items are stored in external files on your hard drive and there is only file path stored in the database. However the path is always the same and is usually around 50 characters long, while the file name is only around 10 characters long. This patch makes sure that only the file name without path is stored in the database, saving some disk space. Clients now also have the capability to understand file names without path in server replies, so we can even reduce size of traffic  between server and clients a bit. I know that 50 bytes is not much, but multiply it by tens or hundreds of thousands of items, and it’s already worth it. As a side effect, all newly created databases will be relocatable, because all file paths are relative, not absolute. There is no plan to make an automated migration to strip the path from existing records, but I might one day implement this in the Janitor, so that users can migrate their database manually if they want to. But it’s not a priority now.

Server-side monitoring

Previously I explained that Akonadi sends notifications via D-Bus to all clients to inform them about changes. The problem here is that not all clients are usually interested in all changes. For instance KOrganizer does not care about new emails and KMail does not care about modified appointments (there are exceptions like the Nepomuk Feeder, which listens to everything for obvious reason). Yet each notification is “broadcasted” to all Akonadi clients. Each client then decides whether it’s are interested in the notification and wants to process it further or just drops it. The average number of clients is around 16, but in most cases only 3 or 4 clients are actually interested in each notification. That means that other 12 or 13 clients just drop the notification. What I’m working on right now is to move this filtering code to the Akonadi server, so that the clients can tell the server what kinds and types of notifications they want and the server will only send notifications to those clients that are really interested in it. This should save us a lot of D-Bus traffic and will fix the awkward situation when all clients are consuming CPU, even when you are just syncing one of your IMAP accounts.

Server-side change recording (IDLE)

The biggest task ahead of me. Some Akonadi clients are using feature called change recording. That means that every notification that is not dropped by the client is stored into a binary file (every client has one such file) and is removed again when the client confirms that the notification has been processed. This is used for example by the IMAP resource. When you are offline the resource is recording all notifications (about items being deleted, moved between folders, marked as read, etc) into the file and when you connect to the internet and the resource is switched to online all notifications are replayed from the file. My plan is to implement something similar to IMAP’s IDLE. Changes will be recorded on the server instead of the clients and all clients will be able to connect to the server and request all pending notifications. After that they send “IDLE” command + explanation of what kind of notifications they are interested in and the server will automatically feed them with new notifications. This is essentially continuation of the “Server-side filtering” feature, but it takes it to a completely new level. With this feature the Akonadi framework will generated almost no D-Bus traffic at all and the whole thing will be much much faster. I’m really looking forward to work on this because it’s a very challenging task and the result will definitely be worth the effort.

Volker has also submitted a few patches to reduce size of the messages sent between clients and server even more and started working on optimizing some SQL queries so that we don’t query the database for data we don’t actually use anywhere.

Of course there are more smaller ideas and improvements in the queue, but I need to keep something for next blog posts so stay tuned – there’s more coming soon! :-)