Table of Contents

Lambda Capturing syntax allows us to quickly “wrap” a variable from the outside scope and then use it in the lambda body. We also know that under the hood the compiler translates lambda into a closure type… but what happens to those captured variables? Are they translated to public data members or private? See the newest guest post from Andreas to understand this tricky problem.

This is a guest post by Andreas Fertig:

Andreas is a trainer and consultant for C++ specializing in embedded systems. He has a passion for teaching people how C++ works, which is why he created C++ Insights (cppinsights.io). You can find Andres online at AndreasFertig.info and on Twitter, GitHub, or LinkedIn.

Originally published at Andreas blog

Capturing variables or objects is the probably most compelling thing about lambdas. A few weeks ago, Bartłomiej Filipek approached me with the example below, which also led to a C++ Insights issue (see issue #347). It was initially raised to Bartek by Dawid Pilarski during the review of Bartek’s C++ Lambda Story book.

int main()
{
  std::string str{"Hello World"};
  auto        foo = [str]() {};
}

The code C++ Insights created for it was the following (yes, the past tense is intentional here):

int main()
{
  std::string str =
    std::basic_string<char, std::char_traits<char>, std::allocator<char>>{
      "Hello World", std::allocator<char>()};

  class __lambda_5_12
  {
  public:
    inline /*constexpr */ void operator()() const {}

  private:
    std::basic_string<char, std::char_traits<char>, std::allocator<char>>
      str;

  public:
    __lambda_5_12(
      std::basic_string<char, std::char_traits<char>, std::allocator<char>>
        _str)
    : str{_str}
    {}
  };

  __lambda_5_12 foo = __lambda_5_12{
    std::basic_string<char, std::char_traits<char>, std::allocator<char>>(
      str)};
}

Bartek’s observation was that the way C++ Insights shows the transformation, we get more copies than we should and want. Look at the constructor of __lambda_5_12. It takes an std::string object by copy. Then in the class-initializer list, _str is copied into str. That makes two copies. As a mental model, once again, think str being an expensive type. Bartek also checked what compilers do with a hand-crafted struct that leaves a bread-crumb for each special-member function called. I assume you are not surprised, but with real lambdas, there is no additional copy. So how does the compiler do this?

First, let’s see what the Standard says. N4861 [expr.prim.lambda.closure] p1 says the closure type is a class type. Then in p2

The closure type is not an aggregate type.

Now, one thing that (I think is key) is the definition of aggregate [dcl.init.aggr] p1.2

no private or protected direct non-static data members

This is to my reading some kind of double negation. As the closure type is a class but not an aggregate, the data members must be private. All the other restrictions for aggregates are met anyway.

Then back in [expr.prim.lambda.closure], p3

The closure type for a lambda-expression has a public inline function call operator…

Here public is explicitly mentioned. I read it that we use class rather than struct to define the closure type.

What does the Standard say about captures? The relevant part for this discussion is [expr.prim.lambda.capture] p15:

When the lambda-expression is evaluated, the entities that are captured by copy are used to direct-initialize each corresponding non-static data member of the resulting closure object

The data members are direct-initialized! Remember, we have a class, and the data members are private.

Captures Fact Check  

The AST C++ Insights uses from Clang says that the closure type is defined with class. It also says that the data members are private. So far, the interpretation of the Standard seems fine. I don’t tweak or interfere at this point. But, Clang doesn’t provide a constructor for the closure type! This is the part that C++ Insights makes up. This is why it can be wrong. And this is why the C++ Insights transformation was wrong for Bartek’s initial example. But wait, the data members are private, and there is no constructor. How are they initialized? Especially with direct-init?

Do capturing lambdas have a constructor?

I discussed this with Jason about this; I think at last year’s code::dive. He also pointed out that C++ Insights shows a constructor while it is unclear whether there really is one. [expr.prim.lambda.closure] p13 says the following:

The closure type associated with a lambda-expression has no default constructor if the lambda-expression has a lambda-capture and a defaulted default constructor otherwise. It has a defaulted copy constructor and a defaulted move constructor (11.4.4.2). It has a deleted copy assignment operator if the lambda-expression has a lambda-capture and defaulted copy and move assignment operators otherwise…

There is no explicit mention of a constructor to initialize the data members. But even with a constructor, we cannot get direct-init. How does it work efficiently?

Suppose we have a class with a private data member. In that case, we can get direct-init behavior by using in-class member initialization (or default member initializer as it is called in the Standard).

int x{4}; // #A Variable in outer scope

class Closure
{
   int _x{x}; // #B Variable using direct-init
};

Here we define a variable in an outer scope #A and use it later #B to initialize a private member of Closure. That works, but note that inside Closure, it is _x now. We cannot use the same name for the data member as the one from the outer scope. The data member would shadow the outer definition and initialize it with itself. For C++ Insights, I cannot show it that way if I don’t replace all captures in the call operator with a prefixed or suffixed version.

Once again, we are in compiler-land. Here is my view. All the restrictions like private and a constructor are just firewalls between C++ developers and the compiler. It is an API if you like. Whatever the compiler internally does is up to the compiler, as long as it is as specified by the Standard. Roughly Clang does exactly what we as users are not allowed to do, it to some extend, uses in-class member initialization. In the case of a lambda, the compiler creates the closure type for us. Variables names are only important to the compiler while parsing our code. After that, the compiler thinks and works with the AST. Names are less important in that representation. What the compiler has to do, is to remember that the closure type’s x was initialized with an outer scope x. Believe me, that is a power the compiler has.

C++ Insights and lambda’s constructors

Thanks to Bartek’s idea, the constructors of lambdas take their arguments by const reference now. This helps in most cases to make the code behave close to what the compiler does. However, when a variable is moved into a lambda, the C++ Insights version is still slightly less efficient than what the compiler generates. Here is an example:

struct Movable
{
  Movable() { printf("ctor\n"); }
  Movable(Movable&& other) { printf("move-ctor\n"); }
  Movable& operator=(Movable&& other)
  {
    printf("move =\n");
    return *this;
  }

  Movable(const Movable&) = delete;
  Movable& operator=(const Movable&) = delete;
};

int main()
{
  Movable m{};

  auto lamb = [c = std::move(m)] {};
  lamb();
}

If you run this on your command-line or in Compiler Explorer, you get the following output:

ctor
move-ctor

This is the transformed version from C++ Insights:

int main()
{
  Movable m = Movable{};

  class __lambda_22_17
  {
  public:
    inline /*constexpr */ void operator()() const {}

  private:
    Movable c;

  public:
    // inline __lambda_22_17(const __lambda_22_17 &) = delete;
    __lambda_22_17(Movable&& _c)
    : c{std::move(_c)}
    {}
  };

  __lambda_22_17 lamb = __lambda_22_17{Movable(std::move(m))};
  lamb.          operator()();
}

Here is the output which you can see on Compiler Explorer:

ctor
move-ctor
move-ctor

Notice the second move-ctor? This is because it is still no direct-init. I need a second move in the lambda’s constructor to keep the move’ness. The compiler still beats me (or C++ Insights).

_Lambdas: 2, Function objects: 0_

What’s next  

On his blog Andreas also has other posts where he goes under the hood and explains how the compiler translates lambda expressions.

See here: Andreas Fertig Blog

Support C++ Insights  

Have fun with C++ Insights. You can support the project by becoming a Patreon or, of course, with code contributions.