QCoro 0.2.0 Release Announcement

Just about a month after the first official release of QCoro, a library that provides C++ coroutine support for Qt, here’s 0.2.0 with some big changes. While the API is backwards compatible, users updating from 0.1.0 will have to adjust their #include statements when including QCoro headers.

QCoro 0.2.0 brings the following changes:

Library modularity

The code has been reorganized into three modules (and thus three standalone libraries): QCoroCore, QCoroDBus and QCoroNetwork. QCoroCore contains the elementary QCoro tools (QCoro::Task, qCoro() wrapper etc.) and coroutine support for some QtCore types. The QCoroDBus module contains coroutine support for types from the QtDBus module and equally the QCoroNetwork module contains coroutine support for types from the QtNetwork module. The latter two modules are also optional, the library can be built without them. It also means that an application that only uses let’s say QtNetwork and has no DBus dependency will no longer get QtDBus pulled in through QCoro, as long as it only links against libQCoroCore and libQCoroNetwork. The reorganization will also allow for future support of additional Qt modules.

Headers clean up

The include headers in QCoro we a bit of a mess and in 0.2.0 they all got a unified form. All public header files now start with qcoro (e.g. qcorotimer.h, qcoronetworkreply.h etc.), and QCoro also provides CamelCase headers now. Thus users should simply do #include <QCoroTimer> if they want coroutine support for QTimer.

The reorganization of headers makes QCoro 0.2.0 incompatible with previous versions and any users of QCoro will have to update their #include statements. I’m sorry about this extra hassle, but with this brings much needed sanity into the header organization and naming scheme.

Docs update

The documentation has been updated to reflect the reorganization as well as some internal changes. It should be easier to understand now and hopefully will make it easier for users to start with QCoro now.

Internal API cleanup and code de-duplication

Historically, certain types types which can be directly co_awaited with QCoro, for instance QTimer has their coroutine support implemented differently than types that have multiple asynchronous operations and thus have a coroutine-friendly wrapper classes (like QIODevice and it’s QCoroIODevice wrapper). In 0.2.0 I have unified the code so that even the coroutine support for simple types like QTimer are implemented through wrapper classes (so there’s QCoroTimer now)


Download

You can download QCoro 0.2.0 here or check the latest sources on QCoro GitHub.

More About QCoro

If you are interested in learning more about QCoro, go read the documentation, look at the first release announcement, which contains a nice explanation and example or watch recording of my talk about C++20 coroutines and QCoro this years’ Akademy.

Initial release of QCoro

I’m happy to announce first release of QCoro, a library that provides C++ coroutine support for Qt.

You can download QCoro 0.1.0 here or check the latest sources on QCoro GitHub.

I have talked about QCoro (and C++ coroutines in general) recently at KDE Akademy, you can view the recording of my talk on YouTube.

In general, QCoro provides coroutine support for various asynchronous operations provided by Qt. Since Qt doesn’t support coroutines by default, QCoro provides the necessary “glue” between native Qt types and the C++ coroutine machinery, making it possible to use Qt types with coroutines easily.

QCoro provides coroutine support for asynchronous operations of QIODevice, QNetworkReply, QProcess, QDBusPendingReply, QTimer and more. Take a look at the documentation for detailed description and list of all currently supported Qt types.

A brief example from our documentation that demonstrates how using coroutines makes handling asynchronous operations in Qt simpler:

This is a (simplified) example of how we do network requests with Qt normally, using signals and slots:

QNetworkAccessManager *manager = new QNetworkAccessManager(this);
QNetworkReply *reply = manager->get(url);
connect(reply, &QNetworkReply::finished, this,
        [this, reply]() {
            const auto data = reply->readAll();
            doSomethingWithData(data);
            reply->deleteLater();
        });

And this is the same code, written using C++ coroutines:

QNetworkAccessManager networkAccessManager;
QNetworkReply *reply = co_await networkAccessManager.get(url);
const auto data = reply->readAll();
doSomethingWithData(data);
reply->deleteLater();

The co_await keyword here is the key here: it asynchronously waits for the reply to finish. During the wait, the execution returns to the caller, which could be the Qt event loop, which means that even if this code looks synchronous, in fact it won’t block the event loop while keeping the code simple to read and understand.

Building RC LEGO with Arduino and Qt

Recently my 4 year-old stepson saw a kid with an RC racing car in a park. He really wanted his own, but with Christmas and his birthday still being a long way away, I decided to solve the “problem” by combining three things I’m really passionate about: LEGO, electronics and programming.

In this short series of blogs I’ll describe how to build one such car using LEGO, Arduino and a bit of C++ (and Qt, of course!).

LEGO

Obviously, we will need some LEGO to build the car. Luckily, I bought LEGO Technic Mercedes Benz Arocs 3245 (40243) last year. It’s a big build with lots of cogs, one electric engine and bunch of pneumatics. I can absolutely recommend it - building the set was a lot of fun and thanks to the Power Functions it has a high play-value as well. There’s also fair amount of really good MOCs, especially the MOC 6060 - Mobile Crane by M_longer is really good. But I’m digressing here. :)

Mercedes Benz Arocs 3245 (40243) Mercedes Benz Arocs 3245 (40243)

The problem with Arocs is that it only has a single Power Functions engine (99499 Electric Power Functions Large Motor) and we will need at least two: one for driving and one for steering. So I bought a second one. I bought the same one, but a smaller one would probably do just fine for the steering.

LEGO Power Functions engine (99499)

I started by prototyping the car and the drive train, especially how to design the gear ratios to not overload the engine when accelerating while keeping the car moving at reasonable speed.

First prototype of engine-powered LEGO car

Turns out the 76244 Technic Gear 24 Tooth Clutch is really important as it prevents the gear teeth skipping when the engine stops suddenly, or when the car gets pushed around by hand.

76244 Technic Gear 24 Tooth Clutch

Initially I thought I would base the build of the car on some existing designs but in the end I just started building and I ended up with this skeleton:

Skelet of first version of the RC car

The two engines are in the middle - rear one powers the wheels, the front one handles the steering using the 61927b Technic Linear Actuator. I’m not entirely happy with the steering, so I might rework that in the future. I recently got Ford Mustang (10265) which has a really interesting steering mechanism and I think I’ll try to rebuild the steering this way.

Wires

58118 Eletric Power Functions Extension Wires

We will control the engines from Arduino. But how to connect the LEGO Power Functions to an Arduino? Well, you just need to buy a bunch of those 58118 Electric Power Functions Extension Wires, cut them and connect them with DuPont cables that can be connected to a breadboard. Make sure to buy the “with one Light Bluish Gray End” version - I accidentally bought cables which had both ends light bluish, but those can’t be connected to the 16511 Battery Box.

We will need 3 of those half-cut PF cables in total: two for the engines and one to connect to the battery box. You probably noticed that there are 4 connectors and 4 wires in each cable. Wires 1 and 4 are always GND and 9V, respectively, regardless of what position is the switch on the battery pack. Wires 2 and 3 are 0V and 9V or vice versa, depending on the position of the battery pack switch. This way we can control the engine rotation direction.

Schematics of PF wires

For the two cables that will control the engines we need all 4 wires connected to the DuPont cable. For the one cable that will be connected to the battery pack we only need the outter wires to be connected, since we will only use the battery pack to provide the power - we will control the engines using Arduino and an integrated circuit.

I used the glue gun to connect the PF wires and the DuPont cables, which works fairly well. You could use a solder if you have one, but the glue also works as an isolator to prevent the wires from short-circuiting.

LEGO PF cable connected to DuPont wires

This completes the LEGO part of this guide. Next comes the electronics :)

Arduino

To remotely control the car we need some electronics on board. I used the following components:

  • Arduino UNO - to run the software, obviously
  • HC-06 Bluetooth module - for remote control
  • 400 pin bread board - to connect the wiring
  • L293D integrated circuit - to control the engines
  • 1 kΩ and 2 kΩ resistors - to reduce voltage between Arduino and BT module
  • 9V battery box - to power the Arduino board once on board the car
  • M-M DuPont cables - to wire everything together

The total price of those components is about €30, which is still less than what I paid for the LEGO engine and PF wires.

Let’s start with the Bluetooth module. There are some really nice guides online how to use them, I’ll try to describe it quickly here. The module has 4 pins: RX, TX, GND and VCC. GND can be connected directly to Arduino’s GND pin. VCC is power supply for the bluetooth module. You can connect it to the 5V pin on Arduino. Now for TX and RX pins. You could connect them to the RX and TX pins on the Arduino board, but that makes it hard to debug the program later, since all output from the program will go to the bluetooth module rather than our computer. Instead connect it to pins 2 and 3. Warning: you need to use a voltage divider for the RX pin, because Arduino operates on 5V, but the HC-06 module operates on 3.3V. You can do it by putting a 1kΩ resistor between Arduino pin 3 and HC-06 RX and 2kΩ resistor between Arduino GND and HC-06 RX pins.

Next comes up the L293D integrated circuit. This circuit will allow us to control the engines. While in theory we could hook up the engines directly to the Arduino board (there’s enough free pins), in practice it’s a bad idea. The engines need 9V to operate, which is a lot of power drain for the Arduino circuitry. Additionally, it would mean that the Arduino board and the engines would both be drawing power from the single 9V battery used to power the Arduino.

Instead, we use the L293D IC, where you connect external power source (the LEGO Battery pack in our case) to it as well as the engines and use only a low voltage signal from the Arduino to control the current from the external power source to the engines (very much like a transistor). The advantage of the L293D is that it can control up to 2 separate engines and it can also reverse the polarity, allowing to control direction of each engine.

Here’s schematics of the L293D:

L293D schematics

To sum it up, pin 1 (Enable 1,2) turns on the left half of the IC, pin 9 (Enable 3,4) turns on the right half of the IC. Hook it up to Arduino's 5V pin. Do the same with pin 16 (VCC1), which powers the overall integrated circuit. The external power source (the 9V from the LEGO Battery pack) is connected to pin 8 (VCC2). Pin 2 (Input 1) and pin 7 (Input 2) are connected to Arduino and are used to control the engines. Pin 3 (Output 1) and pin 6 (Output 2) are output pins that are connected to one of the LEGO engines. On the other side of the circuit, pin 10 (Input 3) and pin 15 (Input 4) are used to control the other LEGO engine, which is connected to pin 11 (Output 3) and pin 14 (Output 4). The remaining four pins in the middle (4, 5, 12 and 13 double as ground and heat sink, so connect them to GND (ideally both Arduino and the LEGO battery GND).

Since we have 9V LEGO Battery pack connected to VCC2, sending 5V from Arduino to Input 1 and 0V to Input 2 will cause 9V on Output 1 and 0V on Output 2 (the engine will spin clockwise). Sending 5V from Arduino to Input 2 and 0V to Input 1 will cause 9V to be on Output 2 and 0V on Output 1, making the engine rotate counterclockwise. Same goes for the other side of the IC. Simple!

Photo of all electronic components wired together Photo of all electronic components wired together

Conclusion

I also built a LEGO casing for the Arduino board and the breadboard to attach them to the car. With some effort I could probably rebuild the chassis to allow the casing to “sink” lower into the construction.

Photo of LEGO car with the electronics on board

The batterry packs (the LEGO Battery box and the 9V battery case for Arduino) are nicely hidden in the middle of the car on the sides next to the engines.

Photo of LEGO Battery Box Photo of Arduino 9V battery case

Now we are done with the hardware side - we have a LEGO car with two engines and all the electronics wired together and hooked up to the engines and battery. In the next part we will start writing software for the Arduino board so that we can control the LEGO engines programmatically. Stay tuned!

Qt containers and C++11 range-based loops

Much has been written on teh interwebs about performance of iterations over Qt containers with Q_FOREACH vs. std iterators vs. Java iterators. However there is [very little][1] about how the new C++11 range-based loops work with Qt containers (or maybe I just suck at Googling, but well here I am…). Today I found out that there is a little catch that one has to be very careful about when using range-based loops with Qt containers.

Qt containers are all implicitly shared classes, which means that copying them is very cheap since only shallow copy occurs. When a shared copy is modified (or rather when a non-const method is called) it calls detach() and performs the expensive deep copy. When there are no other copies of the object (i.e. when the reference count is 1) no copying happens when detach() is called. We say that such instance is “not shared”.

To get to the point - the code below performs equally fast in both cases:

QStringList list{ &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, .... };
Q_FOREACH (const QString &amp;v, list) {
   ...
}

for (const QString &amp;v : list) {
  ...
}

However in the following code range-based loop will perform much worse than Q_FOREACH.

class MyClass
{
public:
   ...
   QStringList getList() const { return mList; }
   ...
private:
   QStringList mList;
};

...

Q_FOREACH (const QString &amp;v, myObject.getList()) {
   ...
}

for (const QString &amp;v : myObject.getList()) {
  ...
}

The difference between the first example and this one is that the QStringList in this example is shared, i.e. reference count of it’s data is higher than 1. In this particular case one reference is held by myObject and one reference is held by the copy returned from the getList() method. That means that calling any non-const method on the list will call detach() and perform a deep copy of the list. And that is exactly what is happening in the range-based loop (but not in the Q_FOREACH loop) and that’s why the range-based loop is way slower than Q_FOREACH in this particular case. The example above could be even simpler, but this way it highlights the important fact that returning a copy from a method means that the copy is shared and has negative side-effects when used with range-based loops. Note that if the method would return a const reference to QStringList, everything would be OK (because const …).

The reason for the speed difference is one peculiarity of Qt containers: they have a const overload of begin() which does not call detach(). Q_FOREACH internally makes a const copy of the list, so the const overload of begin() gets called instead of the non-const one.

On the other hand the range-based loop does not take any copy and simply uses the non-const version of begin(). As we explained above, calling non-const methods on shared Qt containers performs a deep copy. Only exception is when the container itself is const because then the const version of begin() is called and the code will behave the same as Q_FOREACH.

Ironically with stdlib containers (std::vector for example) the situation is exactly the opposite. std iterators are not shared classes so making a copy of an std container always performs a deep copy, but calling a non-const method does not trigger any copying. That means that Q_FOREACH, which always takes a copy of the container would be doing a deep copy in such case while range-based loop, which only calls begin() and end() would not be triggering any copying. Although std containers provide cbegin() and cend() methods to get const interators, there’s no need for the range-based loop to use them, since begin() and end() will always perform equally well on std containers.

To prove my point, here is the benchmark code I used. It’s an extended version of an [older benchmark of Qt containers][2].

#include <QStringList>
#include <QObject>
#include <QMetaType>
#include <qtest.h>
#include <cassert>

enum IterationType
{
    Foreach,
    RangeLoop,
    Std,
    StdConst
};

Q_DECLARE_METATYPE(IterationType)

class IterationBenchmark : public QObject
{
    Q_OBJECT
private Q_SLOTS:
    void stringlist_data()
    {
        QTest::addColumn<QStringList>(&quot;list&quot;);
        QTest::addColumn<IterationType>(&quot;iterationType&quot;);
        QTest::addColumn<bool>(&quot;shared&quot;);

        const int size = 10e6;
        QStringList list;
        list.reserve(size);

        for (int i = 0; i < size; ++i) {
            list << QString::number(i);
        }

        QTest::newRow(&quot;Foreach&quot;) << list << Foreach << false;
        QTest::newRow(&quot;Foreach (shared)&quot;) << list << Foreach << true;
        QTest::newRow(&quot;Range loop&quot;) << list << RangeLoop << false;
        QTest::newRow(&quot;Range loop (shared)&quot;) << list << RangeLoop << true;
        QTest::newRow(&quot;Std&quot;) << list << Std << false;
        QTest::newRow(&quot;Std (shared)&quot;) << list << Std << true;
        QTest::newRow(&quot;Std Const&quot;) << list << StdConst << false;
        QTest::newRow(&quot;Std Const (shared)&quot;) << list << StdConst << true;
    }

    void stringlist()
    {
        QFETCH(QStringList, list);
        QFETCH(IterationType, iterationType);
        QFETCH(bool, shared);

        if (!shared) {
            // Force detach
            list.push_back(QString());
            list.pop_back();
        }

        int dummy = 0;
        switch (iterationType) {
        case Foreach:
            QBENCHMARK {
                Q_FOREACH(const QString &amp;v, list) {
                    dummy += v.size();
                }
            }
            break;
        case RangeLoop:
            QBENCHMARK {
                for (const QString &amp;v : list) {
                    dummy += v.size();
                }
            }
            break;
        case Std:
            QBENCHMARK {
                QStringList::iterator iter = list.begin();
                const QStringList::iterator end = list.end();
                for (; iter != end; ++iter) {
                    dummy += (*iter).size();
                }
            }
            break;
        case StdConst:
            QBENCHMARK {
                QStringList::const_iterator = list.cbegin();
                const QStringList::const_iterator = list.cend();
                for (; iter != end; ++iter) {
                    dummy += (*iter).size();
                }
            }
            break;
        }
        assert(dummy);
    }
};

QTEST_MAIN(IterationBenchmark)
#include "iterationbenchmark.moc"
$ moc iterationbenchmark.cpp > iterationbenchmark.moc
$ g++ iterationbenchmark.cpp `pkg-config --cflags --libs Qt5Core` `pkg-config --cflags --libs Qt5Test` --std=c++11 -fPIC -O3 --o iterationbenchmark
$ ./iterationbenchmark
********* Start testing of IterationBenchmark *********
Config: Using QtTest library 5.4.2, Qt 5.4.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 5.1.1 20150422 (Red Hat 5.1.1-1))
PASS   : IterationBenchmark::initTestCase()
PASS   : IterationBenchmark::stringlist(Foreach)
RESULT : IterationBenchmark::stringlist():&quot;Foreach&quot;:
     48 msecs per iteration (total: 96, iterations: 2)
PASS   : IterationBenchmark::stringlist(Foreach (shared))
RESULT : IterationBenchmark::stringlist():&quot;Foreach (shared)&quot;:
     48 msecs per iteration (total: 96, iterations: 2)
PASS   : IterationBenchmark::stringlist(Range loop)
RESULT : IterationBenchmark::stringlist():&quot;Range loop&quot;:
     53.5 msecs per iteration (total: 107, iterations: 2)
PASS   : IterationBenchmark::stringlist(Range loop (shared))
RESULT : IterationBenchmark::stringlist():&quot;Range loop (shared)&quot;:
     177 msecs per iteration (total: 177, iterations: 1)
PASS   : IterationBenchmark::stringlist(Std)
RESULT : IterationBenchmark::stringlist():&quot;Std&quot;:
     51 msecs per iteration (total: 51, iterations: 1)
PASS   : IterationBenchmark::stringlist(Std (shared))
RESULT : IterationBenchmark::stringlist():&quot;Std (shared)&quot;:
     179 msecs per iteration (total: 179, iterations: 1)
PASS   : IterationBenchmark::stringlist(Std Const)
RESULT : IterationBenchmark::stringlist():&quot;Std Const&quot;:
     53 msecs per iteration (total: 53, iterations: 1)
PASS   : IterationBenchmark::stringlist(Std Const (shared))
RESULT : IterationBenchmark::stringlist():&quot;Std Const (shared)&quot;:
     52 msecs per iteration (total: 52, iterations: 1)
PASS   : IterationBenchmark::cleanupTestCase()
Totals: 10 passed, 0 failed, 0 skipped, 0 blacklisted
********* Finished testing of IterationBenchmark *********

Both Q_FOREACH cases are equally fast because as we explained above, Qt always uses the const iterators and no deep copying happens. Range-based loop with non-shared list performs equally well, because even though it calls detach(), there are no copies to detach from and so no deep copy occurs. However range-based loop with a shared list is over 3 times slower, because detach() here will actually perform a deep copy. The same happens with for loop with non-const std iterators, which is basically just expanded version of range-based loops (range-based loops are just a syntactic sugar for for loops with non-const std iterators). For loops with const std iterators perform equally well as Q_FOREACH, because that is what Q_FOREACH does internally.

To sum this up, when using range-based loops with Qt containers:

Make sure the container is const …

// shared, but const, forces call to QStringList::begin() const,
// which does not call detach()
const QStringList list = objectOfClassA.getList();
...
for (const QString &amp;v : list) {
   ...
}

… or make sure the container is not shared.

// shared and non-const
QStringList list = objectOfClassA.getList();
// call to non-const method causes detach() and deep copy,
// 'list' is now non-shared
list.append(QLatin1String(&quot;some more data&quot;));
...
// calls non-const begin(), but detach() of non-shared
// containers does not perform deep copy
for (const QString &amp;v : list) {
   ...
}

Note that this just moves the slow deep-copying outside of the loop, but the deep copy still occurs. The point is that you need to be careful not to create a new copy of the ‘list’ after it has been detached on line 5, but before passing it to the loop on line 9. Failing to do so would make the list shared again and the loop would trigger yet another deep copy.

I was very excited when range-based loops were added in C++0x and I’ve been using them in some new C++11 code I wrote since then. But in Qt-based code I’ll be reverting back to the much safer Q_FOREACH. While it is possible to have range-based loops as fast as Q_FOREACH as we’ve shown above, one has to be really careful and constantly think about whether the container is non-shared or at least const and use Q_FOREACH if not. For that reason using Q_FOREACH everywhere is much safer for now.

I know that this is not any ground-breaking revelation and many of you probably even know of it, but I hope that it will still be useful for people who are not aware of the implementation details of Q_FOREACH and range-based loops, or just like me did not realize the importance of difference between shared and non-shared container instance.

[1] http://blog.qt.io/blog/2011/05/26/cpp0x-in-qt/ [2] https://blog.qt.io/blog/2009/01/23/iterating-efficiently/