Highlighting error

This came up as a question from a GSoC student. “Why are some functions in vscode highlighted differently?” I was initially planning to dismiss the question as minor highlighting inconsistencies are not a big problem. For educational purpose I decided to explain it properly. This was also a good opportunity to setup my blog.

refreshAll is a Qt signal declared the same way as others bellow it so why is it highlighted differently?

Some editors use basic highlighters based on tokenizer or a simplified language syntax. This is a much faster approach than running full parser and is also more robust. Such highlighters don’t need to know much about code, how it is compiled and what each declaration means.

VScode can use more advanced approach which uses a complete or almost complete parser and takes into account some of the C++ semantics. This can be verified using simple test

int foo;
foo; // local variable
doRefresh; // member function
bar; // unknown symbol

As you can see foo, doRefresh, bar is each colored differently with only difference being the kind of object they refer to. This is only possible if highlighter uses a full parser or close to that.

signals:
    void refreshAll();

    void functionRenamed(const QString &prev_name, const QString &new_name);
    void varsChanged();
    void functionsChanged();

refreshAll is the first signal declared after Qt signals: macro. Without any knowledge about Qt signals: void refreshAll() is not a proper C++ member declaration. This can be investigated by hovering over things to get better understanding of how editor interprets the code.

Without any compiler flag information signals: isn’t anything it causes a parser error. During compilation this would have caused compiler to stop with an error. Since this is only for code highlighting (and probably features like go to declarations) parsing everything perfectly isn’t too important. It is better try to parse as much as possible even if some parts aren’t correct. So instead of stopping with error message parser tries to recover. Another reason why having a good recovery mechanism in parsers used by IDEs is that while the code is being typed or refactored it will likely be partially incorrect until it’s finished.

Parsers used by compilers also have recovery mechanism. When compiling a file it’s convenient to see all the errors. Instead of seeing first one, fixing it, trying to compile again, seeing next error and repeating the process it allows seeing 5 errors, fixing all of them and then try to compile again. This reduces number of required compilation attempts to produce working code. That’s the good case, trying to recover may also result in large amount of errors that wouldn’t be there if first one was fixed. In case of GCC this behavior can be configured using flags -fmax-errors=n and -Wfatal-errors. Other compilers have similar options.

Hovering over refreshAll() doesn’t show anything. Hovering over one of next signals shows method declaration and // signal 1 which is a comment from a file generated by Qt moc. So at that point parser has recovered. Having semicolon after refereshAll might have hinted the parser that broken declaration is finished.

So how to improve this? Running CMake configure using CMake plugin gives VScode information about compiler flags required to parse each file. After doing it the code is highlighted correctly. Repeating the mouse hover test now shows that signals is macro defined by Qt and for refreshAll result is similar to signals that didn’t have the problem.