![]()
Stefan Pölz explains source generators: how they are currently used in .NET, how you can build your own, test them, and make them fast. We’re publishing the recording and the transcript of the webinar that went live on February 23rd.
Introduction
Gael (00:00:03)
Okay, we will start. Welcome everybody to this webinar. My name is Gael Fraiteur, I’m the President of PostSharp and the lead developer of PostSharp and of Metalama. And today, we have a webinar about a topic I love, and this is Roslyn source generators and a new feature, maybe one year old of Roslyn and Visual Studio that allows you to generate code. But Stefan will tell us more about this. Stefan Polz or FlashOver. Why FlashOver, Stephan?
Stefan (00:00:59)
Well, this has been my alter ego since quite some time.
Gael (00:01:02)
Okay.
Stefan (00:01:02)
I used to be a firefighter. This is some years ago, and this a firefighter term and that stuck.
Gael (00:01:09)
Okay. Stefan, you are a senior developer at ADMIRAL Sportwetten in Vienna. You are interested and passioned by clean code, especially in C#, test driven development. You like hacking with Rosyln, meta-programming, source generators. And recently, you spoke at NDC Oslo and many user groups. Before I give the floor to Stefan, the webinar will last about one hour. If you have questions, please use the Q&A facility in Zoom and we will answer the questions at the end. I will repeat the questions, select the questions. Stefan, I think we can start.
Stefan (00:02:11)
Cool. So I’ll take over the screenshare?
Gael (00:02:31)
Yes.
Source Generators
Stefan (00:02:32)
Okay. [inaudible 00:02:33] thank you very much for the introduction, Gael. And thank you much for having me. I’m super, super excited to be here. I have obsessed myself with Rosyln in general and with source generators in specific in the last couple of years. So I’m really welcoming the chance to share almost everything that I know about them. So Gael already introduced me perfectly, so I will skip over that. Let’s have a look what we are going to inspect today.
Stefan (00:03:05)
We have a look at what the source generators are as example. So source generators have been introduced with .NET 5.0. And now since .NET 6.0, the .NET team has actually built source generators themself and shipped them with the latest .NET 6.0 SDK, and this will be expanded in .NET 7.0 already. There’s something in preview already released. We will see what has been added to C# 9.0, which facilitates source generators. Then we are going to have a look at what is the power of source generators, but also what are their restrictions, how they look like.
Stefan (00:03:51)
A lot about tooling, how we can get more familiar with Roslyn in general, source generator in specific, how to debug them. How to apply more recent C# features like nullable reference types, .NET standard 2.0 libraries, some of the restrictions we will hear all about that, different versions. Then we see a bit of a performance issue with source generators that we will then fix as well. Have a look at how we could publish our own source generators via NuGet as well. And at the end, there is some more examples and the Q&A section, and also a little bit of a surprise from Gael. I’m really looking forward to that. So let’s get started by example.
Stefan (00:04:42)
In .NET 6.0, which has been released in November of last year, the team actually built in suite generators themself. This one, which facilitates System.Text.Json; another one, which is simplifying the use of logging and actually also the Razor C# file are generated with [inaudible 00:05:15] by now. So, let’s jump in right into code.
Stefan (00:05:18)
The first example I would like to share is… Let’s zoom in, okay, is logging. So here we have a simple usage about we are creating a logger with the minimum log level of TRACE. And we print Hello, World to the console. Let’s see if it’s actually working, and it does. Hello, World is printed. Now we can also parameterize this logging. For example, let’s add name here and let’s add this now to the parameter, so put world here. And this code now still produces the very same output. However, now, there’s a little bit of a caveat that we need to be aware of.
Stefan (00:06:16)
Let me also show this by not passing a string here, which is reference type, but actually passing an integer here, which is a value type. I will enter 0x_F0, which is basically 240. So let’s see if we get hello, 240 in the console. We do indeed. But there is something to be aware of here, we basically caused a little bit of a performance hit because when I hover this log information, have a look at the definition of this extension method. We see that it takes a params of object array.
Stefan (00:06:56)
Let’s have a look at this in SharpLab, this will depict the problem. So I have here this method, I do a params object array, those are my args. Let’s create a caller, public, void, caller. And this caller now calls into the method M with 1, 2, 3. But what is actually going on under the hood now? We see this on the right side. Here we create now, we actually instantiate this object array, the params is just under air quotes, just a C# feature. This is then a compiler, something which can actually be run. And we are more or less hidden creating here an object array and setting these.
Stefan (00:07:50)
Now, since this is an object array, and we are assigning value types, the integers, those extracts, we also have boxing involved. So here we have some hidden locations. Now to improve that… Sometime ago with Microsoft extensions logging, there is also the concept of high performance logging. This is achieved via the LoggerMessage.Define method. So let’s build a helper class. I call it Log. And in here, I now can define this message. So I have my static action, to this action, action, where we first passing the logger.
Stefan (00:08:51)
Then we pass in the parameters that we have, for example, an integer, and there is always an optional exception as well, mark it as nullable. I call this hello. Now we must say LoggerMessage.Define. And we define this so this is one parameter, the integer, and we can now define the log level, I do LogLevel.Information. And let’s say, I want to pass in this 0x_F0 here, which is 240. And now the format of this message I just paste it in here. Now maybe I want to call it to be an extension method. So I make a public static void. I call it, “Hello” and invoke this delegate that we’ve built here. So this is an extension method. So I say this ILogger, and we invoke that with of course… We also want to pass in the parameter, so this is my integer, I call this the number and passing in as well.
Stefan (00:10:25)
By the way this here, this is the event ID, so every log message also has an event ID, And we don’t have an exception so we can pass in now. Now we can invoke this extension method so we can [inaudible 00:10:43] Logger.Hello and pass in our number. Let’s use a different number, let’s use nine. So we now should have the same result. This time it should not print Hello(9), let’s actually confirm that. Indeed, we have Hello(9). So it’s doing the same, but we got rid of both the object array allocation and the potential boxing for this params array.
Stefan (00:11:16)
But this is now quite some code that we always have to write in order to have this performance improvement. So now comes one use case of a source generator. So there is two prime use cases. One of them is to reduce the implementation of repetitive code. Since this high performance logging pattern is always the same, I need to define my message and then actually invoke it. This could be done via a source generator. So the team has introduced source generator for that. I will grab that example. I will replace that. So we now have our log message here, and that basically… Let’s change this to an integer. [inaudible 00:12:11] and we define that, basically we control that via this Logger.Message attribute.
Stefan (00:12:26)
So I still do the very same thing. I have the 240 as an event ID, the LogLevel is still information and it’s still the same format of the string. And let’s see if we invoke that if we indeed get the same result. And we do, we still get Hello(9) but with much less code. And this is now actually generated by source generator, but where is this code? Currently have a back in my tooling you actually get a compiler error in the ID, but actually, we saw it running so it works. But where is this code? I can actually see that when we go into the solution we have, unless tooling doesn’t show… Well if my tooling doesn’t show this, but perhaps I can view the build output. Let me quickly grab that, that’s not it. There we have it. So this is what’s actually generated by a source generator. We see this is auto generated and it basically does the very same thing it defines this action via the LogMessage.Define Method. And also wraps this in an if logger enabled, which is always smart to do always wrap your log message if it is enabled, because if we set up a higher minimum low level than the message is, then we don’t want to point it in the first place and we can avoid the invocation. And yeah, so this is one use case of source generators to eliminate repetitive code.
Stefan (00:14:10)
Another use case is to increase performance. For example, we have System.Text.Json, so Json serialization and deserialization. I will copy in two pieces of code here and here. So what we are doing is with .NET 6.0, new methods have been added to the Json serialized and deserialized methods, which do take this JsonTypeInfo object. And this can also be generated by a source generator. So we define a partial method here… Oh, sorry, a partial class, which derives from JsonSerializerContext. We tell the generator for which type it should be created and can also pass in some options, such as that the output should be written indented. Here, we define the entity that we want to serialize and deserialize and then we do a little launch trip. So we create the entity, we serialize it to Json, print it to the console, and then we deserialize it again. Again, print it to the console.
Stefan (00:15:32)
And let’s see what’s the output. And we do get… Well, I do have a fail here because I put the log level two error. So we see that the first white, this indented deserialized entity, and then again, the serialized [inaudible 00:15:53]. So this is the record two string that we see down here. And so the source generator now here generates a lot of code. We can perhaps see that, not sure what’s wrong with my tooling, but actually there is a bunch of code. I’m just opening one file because there is many. So this is like one file where the serializer options are defined and this default deserialization context which carries this type. So both this default serialization context and this entity type they’re both generated, and there’s a lot of code. I will open another file and another file.
Stefan (00:16:44)
So this is all generated and the benefit now is that general or previous system text Json without a search generator needs to do reflection. So it goes at one time needs to figure out, okay, what’s the entity type. It has a string which is called name. It has a number which is called number and has to figure this out at one time. Well, they then cache the information. So it’s only the first invocation, which is still a bit more expensive, but then this cache requires memory that lingers around in our applications. And source generators now do a very direct and already at compiled time evaluated serialization and deserialization, which now avoids this startup cost so we have improved performance, but also reduce memory allocations. Those are those two very powerful use cases about source generators. So we can get rid of tedious patterns to implement, but also potentially increase performance by most likely reducing System.Reflection. So now we saw this partial keyword being used a lot, but this is actually C# 9.0 syntax because before C# 9.0, if I do… Actually let’s show that in visual studio, if I would, let’s get rid of that, create a public static, partial class, call it Helper. And in there a partial method, I have it here, I call this method Get. So, before C# 9.0 this wouldn’t have compiled. Well, I do get a compiler error here, but I’m not sure what’s wrong with the tooling, but this wouldn’t have compiled because the explicit accessibility keyboard was not allowed. Partial methods so far have always been implicitly private because if there is no implementation to it, then the compiler basically strips it from the compilation. So this is also why it couldn’t have return types. What wasn’t also allowed out parameters, haven’t been allowed in partial methods. Now with C# 9.0 they are allowed but if I now enter an explicit accessibility, such as internal, now the new rules, the C# 9.0 rules apply and they state that it is okay return something, out parameters are okay.
Stefan (00:19:47)
But now the implementation must be provided. If the implementation isn’t provided, they get a compiler error. Now the tooling actually fires this compiler error. Not sure what’s wrong with my setup it seems that it can’t find my source generator but it’s actually there because I wrote a source generator, which looks for this partial method and actually provides an implementation for it. And let me show that. So if I do via the logger… Or let’s just do on Console.WriteLine and I say Helper.Get. And if I run this, Console, dotnet run, we actually get printed, “From Generator”, but we haven’t written this. This is coming from a generator.
Stefan (00:20:37)
Let’s have a look at the generated file, looks like this. So this generator now creates the second part. Let me put this side by side, actually. So the generator creates the second part of this partial class and creates the implementation to this partial method. So this implementation with the usage of [inaudible 00:21:00] must be provided, but it may provided by a source generator.
Stefan (00:21:04)
Another C# 9.0 improvement is module initializers. If I add a bit of code here to this helper class, we now have the module initializer attribute, and this is called whenever the module initializes. Most of the times assembly and module, or at least in my mind, I’m not entirely sure what the difference between assembly and a module is, but basically when the assembly loads, then this one is initialized.
Stefan (00:21:35)
And I can improve that if I call Helper. this text property. We have no code which actually invokes this Init method here. We see this as 0 references, but still… I close the console. If we want this, we get this initialized string that we actually set here. We never invoked this method, but this is done with this new C# 9.0 features, module initializers. So our source generator could, or with module initializers we could ensure that something is initialized, that source generators require. And those two features now may be heavily consumed and used by source generators.
Stefan (00:22:53)
Now, what are the laws of source generation? A source generator gets as an input the compilation of the user. So basically the user code, all the code that I have in my solution, where I do include the source generator is basically put, well, not all of it, but most of it is put as an input to the source generator. For example, the C# code is passed to a syntax tree. This is very similar to other Roslyn extensions, such as analyzers or code fixes. As a matter of fact, a source generator, technically speaking is a Roslyn analyzer as well. So the programming model is very similar. So we get the syntax trees, we also get the semantic model that we can ask questions about is this identifier entity? What type is it? What members does it have?
Stefan (00:23:56)
One limitation is we can only add new sources to the compilation. And those new sources are actually added as strings, so the input is the syntax tree semantic model, additional files and so on. And the output is basically a string. And this string now gets put back to the compilation and will be part of the final compilation, which will then be put into the DLL if we build it or into the NuGet package, when we pack it. And we can only add new sources, a source generator cannot modify or remove existing code it can only add to it. So this is how this adds up with the partial basically extension point.
Stefan (00:24:44)
A source generator can also produce diagnostics because technically speaking a source generator is an analyzer. It could produce warnings, for example, if I misuse the source generator then we could produce warnings to indicate to the user and guide them how to use them correctly. We can also access additional files. For example, we could mark a text file as an additional file. In the source generator, we could have something like a transpiler, which then spits out C# code from this additional file, whatever it is. Maybe we could transform Json into some C# initialized dictionary, perhaps.
Stefan (00:25:33)
Source generators are un-ordered. So if you have multiple source generators, well, they do run in a particular order, but this is undocumented so this may change. Source generators cannot depend on each other so each source generator only sees the same input. They do not see each other’s output. And I already said that technically analyzes. And we went through the use cases, avoid boilerplate code, avoid Reflection for-
Stefan (00:26:00)
The use cases. Avoid boilerplate code, avoid reflection for better performance. In this presentation, everything you see is also available on GitHub. I put in documentations, also the cookbook which is from the .NET team, delivers some recipes on how to solve issues with source generators. Now what’s the anatomy of a source generator? How does it look like if we want to build one? The most important thing is… We need to include in our project, which has to be a .NET standard 2.0 project, because source generators also need to run in Visual Studio. That’s still based on .NET framework. So for compatibility reason, just as any other Roslyn plugins, such as code refactoring or diagnostic suppressor, they have to be published as standard 2.0 packages. And this project then needs to reference the Microsoft.CodeAnalysis C# workspaces. Maybe let me switch to… Red should be better visible. Yes. So this includes everything we need, to build a source generator. This has been added in version 3.8.0, which is basically .NET 5. So this is only compatible now with .NET 5 and higher. So source generators don’t work with .NET Core 3.1 or .NET Framework. We could consume it in a project which targets .NET Framework, but this project needs to be built with the .NET 5 SDK.
Stefan (00:27:57)
And additionally, I always like to add this set of analyzers. So this is basically an analyzer which analyzes the source generator that I built. So some sort of meta analyzer, because it analyzes my analyzers. Always add them as private as at all, because we don’t want these dependencies to transitively flow to the consumer. And then we create our generator. Our generator derives from IsourceGenerator. We need to define two methods. We have an initialized step where we basically set up the source generator and then the Execute step, which eventually could produce, it doesn’t have to, but could produce a C# string. And we need to put the generator attribute on top of the type so that the tooling, the driver, is aware of it. Additionally, we can define a SyntaxReceiver. Actually, let me show this in action.
Stefan (00:29:06)
So here I will briefly show another example of a source generator. Examples, yes. If you have seen the talk from David Fowler, Implementation details matter, he showed the example of how in on the to string is a little bit slower than potentially anticipated because in the background it does reflection. And this reflection or this cache reflection values are then looked up in a binary… Be a binary look at been in a way. And so this is not this not as fast as it can get. So one suggestion is to actually have a source generator, which creates a more naive approach, which is basically just a switch. Overall the members of that Enum, and then basically do a name off and print this out. And this is significantly faster than doing a regular Enum to String.
Stefan (00:30:20)
So how can we create this code? So here, everything is generated. This Enum info type is generated and all its members. Oops, excuse me. And all its members are generated as well. So we see again, the auto generated on top. When we generate a file always put the auto generated on top because this per default suppresses other analyzers to try to analyze our generated code because the user has no direct control over how the analyzer creates these. And we see in the tooling that Visual Studio shows this file is auto generated by that generator, you cannot edit it. Let’s jump to that Enum input generator.
Stefan (00:31:21)
We see it needs to write from my source generator and it has the generator to build on top. Now in the initialized context, this optional, we can register a SyntaxReceiver via the register for syntax notification method. And the SyntaxReceiver now is a type, it arrives from SyntaxReceiver, and it has one method. It has the on visit syntax node, and this gets in each and every syntax node in the compilation. So basically the user’s code. So just to give a bit of context… What’s a syntax node? I will again, jump to SharpLab
Stefan (00:32:11)
And exactly this code. How would this look like from a syntax three point of view? I can switch to the Syntax Tree. And now we see the Roslyn representation of that code. We see this [inaudible 00:32:24] compilation unit. This is now our tree. We have a using directive, which is here. We have our class declaration, which is this entire type. Now this class declaration, it has a public keyword. It has two method declarations, from where I jump in then. We will eventually find out that the first method declaration is this color method, which return type is a predefined type. This void keyboard… Actually, as a matter of fact, System.Void exists, it’s an unspeakable type, but it’s basically there.
Stefan (00:33:06)
And the second method is our other method, which has a parameter list. And in this parameter list, we have several parameters. One parameter has the params keyword and so on, so forth, we can keep going and expanding the tree. And each of these things that I can expand, so this is each node. And those nodes now get supplied to this SyntaxReceiver on visit syntax node. And now we can do filtering. So in the source of arena, I basically… What we want to achieve… If I jump back to this example, what we want to achieve is to find every invocation of the Enum info GetName method. So we find all the GetName methods, which are invoked to form parameters. They are candidate for what we actually want to generate source code for.
Stefan (00:34:08)
And this is basically what we do. So we check is this SyntaxNode, indeed, an invocation expression. So calling for example, calling a method. We have a look at this invocation. If it has indeed one argument, then it’s a good candidate. And additionally, we check if this expression… If it’s a member access expression, could be something else. And we also check the name. So behind this method name, actually hides ticket name that we want to generate for. And all of those candidates, we can then expect further. We basically now get this argument expression. So we extract the Enum constant that we invoke it with. So we basically extract this information and then put it into a list. And this list cannot then be reused by the rest of the source generator. So if the SyntaxReceiver, we can narrow down all the SyntaxNodes that we get to just a specific set of nodes that we may be interested in. And then comes the Execute step. So in the Execute step, we can now generate our source code. Let’s have a look at the parameter, the generator execution context.
Stefan (00:35:39)
So the generator execution context now holds all the information that we need. It has the compilation, it has pass options. For example, in the pass options, we can find out whether this project has been compiled with C# 9 or C# 10. There could be additional files. If we mark them with additional files to the compilation, there is also config options. So we could provide settings, for example, via a editor config or a global analyzer file, or also via MSBuild. We could mark specific MSBuild properties and control the generation via that. We have our SyntaxReceiver, that we had a [locum 00:36:21] and also cancellation token. So since the generator takes time, it’s part of the compilation. If we want to abort it, if we have a longer loop, we should from time to time try to cancel, if cancellation has been requested.
Stefan (00:36:37)
And now comes the meat of it. Here is our add source method, which takes a string, the hint name. This will be then shown in, for example, in Visual Studio per default. Source generated files are actually not put to disc. They’re just virtually there. There is an option to change that, for example, for debugging, but in general, they’re not available. But we can supply a hint name for the tooling to show this, for example, up here. And then we have our string source, which is actually this C# text that we want to add to the compilation. So there’s literally a string. It needs to be a string, a valid C# string, otherwise the compilation then would fail. So the generator could successfully generate invalid C#, which then would in the second step after the generator’s finished, fail the compilation.
Stefan (00:37:40)
And with that, I now highlight the most interesting portions, basically. First we see… We know we have a SyntaxReceiver. This is our Enum info SyntaxReceiver. I also check via the pass options is this C#. We can do this. This is this like standard Roslyn. We can check if this language is indeed C#. I also get the options, so this analyzer could also via the analyzer options… But let’s have a look at the generate source code. What I like to do is to use an indented text writer, so I can add tabs to the source code that I actually generate, but it could be any strings. There is different means. You could use a plain string builder or use other means of creating the C# string, maybe a more template based approach. Everything works. What we need is a string.
Stefan (00:38:42)
I always like to add the auto-generated on top. That’s the first thing. What we also should keep in mind is the language version, because we could technically generate a C# string in our generator. But what if the consuming project has its language version set to C# 9 or perhaps C# 7.3, because it’s a .NET standard 2.0 project, which is the default. Then for example, with C# 7.3, nullable reference types are not available yet. So nullable enable wouldn’t be meaningful, it actually would cause a compiler warning if this feature isn’t there. So always check… Well in this analyzer, I actually check, is this indeed at least C# 8. And if it is, then we add the nullable enable. So keep in mind about what language version you want to support. Then we add a name space, we add the class and so on and so forth. So basically now compile our string that we then want to spit out to the compilation. And eventually we to string this and at the very end… Then add this to our compilation. And then we have it in here and we can actually F9 it, F12 it. So actually let’s put this side aside and see the generator in action, because if I start removing stuff from here… Now on the right side, this actually gets updated. So if I remove everything, this here is the plain minimum that this generator generates. It basically just has a placeholder to enable intellisense, so at least generates this Enum info type with a GetName where we can feed in any Enum, but we throw an exception. This should actually never happen because if I start typing, if I say Enum info .GetName, and now pass Enum, let’s use a StringComparison.Ordinal. Let’s use ordinal. And now the generator picked that up. The SyntaxReceiver found this candidate and the Execute step, then added the source. And there we have it, a more performant way of how we could do basically to string, our Enums. Perhaps do I have… Did I want that in advance? I do have a… Oops.
Stefan (00:41:46)
I must have deleted it by accident, but I did want to branch off. Perhaps wanted it at the end, which actually shows now that this source generated approach is indeed faster than the basic to string. Now let’s come to some tooling which could help us building source generators. Oops. I already showed SharpLab with the Syntax Tree. This is also available in Visual Studio. If I jump into Visual Studio, let’s have a look at the SyntaxReceiver file. I have here the Syntax Visualizer. I do get that if I open the installer. Go to modify and select individual extensions of work. Is this now enabled by default, actually? I did not expect to see the… It’s called the .NET compiler SDK. So this is… Somewhere that we need to select this, not installed per default. I can’t find it right away, but it’s under this Visual Studio extension workload. And when we install this, we get the Syntax Visualizer and this shows now the same representation that SharpLab does, but basically now a code. So I can click here. If I click this field declaration, then it gets highlighted. So I see this is a field declaration then I can jump into it. What are the nodes? How is this tree assembled? Not a very handy tool is the Roslyn quota.
Stefan (00:43:53)
This now goes in the other direction. So if you would like to find out the other way. So if I put in a string… Let’s say public static class, my class. Now what would I need to do? Roslyn style in order to create this tree. So I can now generate this code. And I see down here, this is basically what I need to call via Roslyn in order to create a Syntax Tree, which we present this string up here. So I create a compilation unit, which has the class declaration. And this class has to modify us the public keyword and distract the keyword and so on. So this will go in the other direction.
Stefan (00:44:41)
And there is also a source browser. Perhaps you are familiar with source.dot.net, which is basically the source browser for the top net one time repository, where we can find types such as the cancellation token that I’ve mentioned previously. And we can then go to web access and directly jump to the source on GitHub. And this also exists for Roslyn, so there is sourceroslyn.io. And if I would search the ISourceGenerator interface, we see it. This can also jump to the webs access and we jump to the Roslyn repository and see how actually the compiler team is achieving all of this.
Stefan (00:45:39)
Now I would like to show a bit about debugging. So in Visual Studio, we can mark a project. There is Roslyn component, and also we can emit those. This is basically what I showed, what I would expect is actually that, maybe it works in this project. If we have a look at the… Close that. At the example, you see this example now consumes the generator.
Stefan (00:46:20)
In this case, I also need to set because I don’t reference it as a new package, a reference to project directly. I need to tell MSBuild that this is not a library, but this is actually a wasting component. So I say, I don’t want to reference the output assembly because there is no assembly I want to reference. I just want to enable this analyzer because Source Tree are technically analyzers. And in this case, I also set target framework. So this is how I can form an example project, consume my generator directly. And if I go to the dependency tab… Let me open it and then show it. Yes. We see that here on the dependencies, we have analyzers. This is the FCU generator that we have had a look at. Here is the Enum info generator and the Enum info generator generates this enuminfog.cs. So what you see here, this is this hint name that we give to the tooling so that we can display it here. And I can double click and then inspect what is actually generated. But also I can put this out physically to disk. So if I jump to this directory…
Stefan (00:47:42)
We can go to the obj folder and have the generated directory in here. Again, my generator, the info generator interior is the same file. But now here’s the physical file, the physical representation. How this is enabled is via [depository flex 00:48:02]. We can emit those compiler generated files and then we can tell MSBuild what the generator… What a tooling… Where to put them, where compiler generated file output path. I say the base intimate audio output path which is the obj folder, generated folder. And then we will find our generators in here.
Stefan (00:48:23)
In order to enable debugging in Visual Studio. We can also set a debugger profile. We set this to the debug Roslyn component and can then actually debug into source trailers. Let’s see if this is working. So I will, if I have a long… So we see here, this is the generator project, our top net standard 2.0 source generator. Here are the launch settings and in the launch settings, I now say the target project. So the project, which should be executed in order to run this generator. Tell it to use this example project. And let’s actually select this now as a startup project generator and do a break point in our Enum info generator.
Stefan (00:49:27)
Perhaps let’s just before we emit it. And there we are, and we see the hint name, it’s this enuminfog.cs. And the source text, let’s shift + F9 that. So here we have it, this actually also has text. Here is the string that we will output to the compilation. So this may help with debugging and stepping through code. There’s also another way via unit testing, this is actually my preferred way. If we have unit tests, I can also debug. So actually let’s keep that. Break point, if I go to my Enum info generator test…
Stefan (00:50:22)
There it is. Oh, this is actually the generator test. This one. Yes. So if you’re familiar with testing in Roslyn, the usual approach is to define an input string, and then we can expect an output string. So let’s jump to for example, this test. I do have here the string comparison, and I call this EnumInfo.GetName(). So this is basically the input text. And as an output text, I now expect what the source generator creates.
Stefan (00:51:04)
So indeed for this string comparison, this switch case that we want to generate. And we can then with this tooling, insert that later on, we verify, and we can also just debug this test. To see that, we will jump in here as well. So debugging works from a unit test in my opinion, better than via the debugging experience, because this also works for wider, for example. And it’s always good to have a unit test. Speaking of unit testing, how can we achieve unit testing? So there is a package for that. There is in the Microsoft.CodeAnalysis, Csharp.SourceGenerators.Testing.XUnit. There’s the xUnit flavor. This also exists for MSTest and NUnit. If we want to find out more about this also.
Stefan (00:52:00)
And if we want to find out more about this… I also linked this all in the presentation. But basically, if we search microsoft.codeanalysis.testing, this leads us to here. So this is in the Roslyn SDK project on GitHub. And with that, we can test analyzers, code fixes, also code refactorings and here our source generators. So with these packages, we can nicely unit test our source generators. And also, what I showed already is from the consuming project. We need to set it like this with… If it’s packed in a NuGet package, the most important part is to keep in mind to have always private assets all.
Stefan (00:53:06)
Speaking of NuGet package, I’m actually doing this right away. How does a generator look like in NuGet? So, let’s have a look at nuget.org. Let’s jump to the generator that we showed, F0.Generators, and inspect this in the in NuGet Package Explorer. Now, we see what’s actually in this package. So, we see we do not have a lib folder. We could have both. For example, the microsoft.login.extractions, this both supplies the library plus the generator. Here, this is a generator only. So what we have is since generators are technically analyzers, we see here Analyzer in .NET CS. Here is the generator. And since this is not the library, we have no dependencies or no target framework dependencies. Since this is a generator, we need at least .NET 3.5 SDK in order for the source generator to work.
Stefan (00:54:20)
Talking about nullable reference types. So, one thing about source generators is this actually applies to all Roslyn projects. They need to be .NET Standard 2.0. But the default language of .NET Standard 2.0 is C# 7.3. But for nullable reference types, we do require at least C# 8. So what I like to do is always multi-target the analyzer to always target the latest .NET target framework so that we get the latest nullable annotations. We enable nullable. But now, we need to be careful because an analyzer must be a netstandard2.0 project. So this is then for the NuGet package… I will show this also then in a second. We need to select the correct target framework. In order to get the nullable attributes, we can use this package, the nullable package. And for 2.0 target framework, we should actually suppress nullable warnings because the BCL is not annotated yet. So, it wouldn’t be very meaningful.
Stefan (00:55:31)
Now the Rosyln APIs, they actually are mostly annotated. Are they mostly? This is a lot of code or at least a lot of it is already annotated. So, these are fine. For the .NET 6 build, we would use that to get our nullable annotations right. But in netstandard2.0 that we do define as now oblivious basically, we wouldn’t mind because we already checked it for .NET 6 and we have warnings as errors. Then we are fine, good to go. Now, I want to say a bit about versioning. So, what I showed so far is what has been added in Roslyn 3.8 which is basically the .NET 5 SDK. It requires at least Visual Studio 16.8 or Rider 2020.3. Now, in Roslyn 3.9 which is the .NET 5.200 SDK, which is a readable since Visual Studio 16.9, we have a new feature for source generators. Let me briefly show that. So here, we now have a 3.9 generator. So this is basically how we control what feature set, what language set we have, what compatibility level depending on the package that we consume. It’s still a .NET Standard project. And what we have now is something called the post initialization. So, we can register for post initialization and put out source code basically unconditionally. Now, this generator post initialization context, it is much slimmer than the other one. We have no compilation here, no pass options, no additional files. So, we can basically do unconditional generation of codes that we always want to generate. And now, the thing is that other generators may depend on something which has been emitted by post initialization context. Everything which is emitted in the Execute step, where if you have motor generators cannot depend on each other. But everything which has been post initialized maybe there.
Stefan (00:57:55)
Actually, have here the evidence. So, I have here a syntax node receiver. Actually, where is it? There it is. So in the Execute, we have a look at the compilation. We get this post initialization type that we do create here in the post initialization name space. We create a type called Roslyn 3.9 and we can actually assert that this is not null. So, this really exists. And then basically say, “Okay, this type has really been found.” So, let’s see if this actually is there. If we again have a look, [inaudible 00:58:35] analyzers. Here is our… Oh no, in the example of course. Here’s our demo analyzer and here is this post initialization file. And this one we put out unconditionally. And the other one now says, “Okay, the post initialization step is found. It is available.”
Stefan (00:58:59)
Now, while I have this open, I also want to show a bit of a potential performance issue because source generators may be invoked many times. So if I go to the demo, and I hope we can see this if I… Yes, it works. So, I hit Control D to duplicate this line. And we see on the right side that more and more nodes are visited. So, the more code we have. So this counter is actually, if we have a look at the generator, we do have a syntax receiver here and this syntax receiver just increments the node visited. It counts every time it receives any node. And then, we just put this out in this comment, rewrite it here. And this is what we see there that the more code we have, the more often we invoked this method. And this is now invoked for basically every key press. So if I start typing a comment, now this is invoked. And if I add more code or add a new file and the more I type, the more it is invoked.
Stefan (01:00:07)
And if we have a small project, this is no problem. If we have a big project, it’s most likely still not a problem. But if we have a huge project, for example, the .NET runtime repo, now this may cause huge performance issues. And actually, since this is [inaudible 01:00:24] Visual Studio is actually also called a Visual Studio, then may decrease the editing experience. And we may now experience lagging between our key presses because the generator or Roslyn NET generator kicks in all the time and with every key press emits everything new. And this is not very fun to edit with.
Stefan (01:00:46)
And this performance issue has been addressed with the latest version of source generators which have been added now with Roslyn 4.0 which require at least the .NET 6 SDK, which in turn at least is available in Rider 2021.3 or in Visual Studio 17, which is Visual Studio 2022. Which means now if we will upgrade our source generators from 3.8 to 4.0, this no longer works with the .NET 5 SDK. We require at least the .NET 6 SDK. But since I think .NET 5 is running out of support in May, I guess this is fine to do depending on your use case. Now, how does this .NET 6 style generators work? I can show you another project. This is yet another analyzer that we have a look at. For example, I will briefly show the example before. Let me change here something. Let’s change this to… No, let’s show this first.
Stefan (01:01:57)
We have here a small program which uses a record class. However, a record class, so that is actually not a record, the record feature itself, but the init keyword that a record is utilizing. If we have a look at #lab, I’ll put in this record here. Have a look at the C# version of the code. And let’s move that. Yes, we see that in the generated code this property that we had defined here uses the init keyword. It is compiler generated. But in order for this to work, we need that IsExternalInit type available which is this one. I have 12 of this. So, this is available in the system one time. Actually, let’s have a look since when is this available? This is available since . NET 5. So, I couldn’t use a record type in something lower than .NET 5. Let me change this to NET 472. So, I’m now switching to .NET Framework.
Stefan (01:03:15)
Now, .NET Framework doesn’t have this type available. And although I set the language version to default which is… I’m using .NET 6 SDK here which is C# 10 and I build, this project still compiles because I have a source generator now supplying this type. If I have 12 of this now, we will see this now is no longer the type from .NET, but this is now source generator type. So the compiler, basically it just needs this type available in order for the init only feature to work. And how does this generator work? This is now actually the newer version, a Roslyn 4.0 [C Node 01:03:59] Generator which is called an incremental generator. And the programming model with incremental generators is a little bit different than with regular source generators. Under the hood, an incremental generator, as the name suggests, tries to do stuff incrementally. So, we build up a pipeline which then gets executed.
Stefan (01:04:28)
And during this execution of the pipeline, the driver of the generator figures out which parts have already been executed and if they produce the same output as before because if the same output as before… Or if they receive the same input as before. Because if they receive the same input, we don’t need to produce all the output that we already cached before again. So with the combination of this pipeline and caching, incremental generator now can improve the editing experience for Visual Studio so that we don’t generate everything and you only because we hit semicolon somewhere. Now, to mention that if we do a .NET build or want to see I system, there won’t be any difference between using… Or no noticeable difference between using I source generators or incremental generators. But from an editing experience, depending on the size of the project, the experience may vary tremendously.
Stefan (01:05:33)
So with an I incremental source generator, we only have one single method. We have initialized method. So the syntax receiver is gone because the syntax receiver, this was or could be the potential performance issue because it gets a lot of nodes all the time. So, this is gone now. This looks now a little bit different. The compilation and the past options and the additional texts, we don’t have them available directly but via this incremental value provider. And this is the core type that we use in order to build up our pipeline. So first, we have a syntax provider. We can create a syntax provider by feeding in two callbacks. We first have a predicate. This basically returns bool which says, “Are we interested in the nodes?” So, this is now the comparable part or the equivalent to the syntax receiver.
Stefan (01:06:34)
Let’s have a look at this method because this method now again gets in the syntax node just as the on syntax node visited from the previous syntax receiver. And returns bool saying, “Is this node interesting for us? Could it be a potential candidate for a generator?” What you do in here, we see is this a record declaration? And does this record declaration has at least one parameter? Because then, init only property is admitted. The alternative would be if we have an access declaration syntax, basically if we define the init keyword ourself. So here, we have the explicit usage of init. Down here, we have the implicit usage of init. And if it does, then this node is interesting for us. Then we actually do have to emit this type if it’s not available. The second method to this syntax provider is the syntax transform which now basically receives the node. And we can return pretty much anything.
Stefan (01:07:48)
We could transform it to whatever we need in order for our pipeline then to be meaningful. In that case, I basically just return the node. But we could turn it, for example, into a string or a collection of something. And then, we combine that step with the next step of the pipeline because now I need to find out if this type that I potentially need to generate is already there. So basically, do we have a target framework which already defines this type? What made this type… Is external init type already defined manually? And for that, we need the compilation. But we don’t have the compilation directly. We only have in the context a compilation provider which is incremental value provider. And this is now where the pipeline kicks in. So, we combine that. We combine the compilation provider with our previous step. And then, we get a combination. So, this is actually this many tuple here which now holds the collection of our syntax nodes and the compilation.
Stefan (01:08:51)
I also want to see what language version this is. Is this actually a language version which supports the init only keyword? Because it doesn’t make sense to emit this type. And for this, I need the parse options. So again, I now get the parse options provider and combine that with the previous step with this tuple up here. And once I have combined, I could now also combine this with more. For example, in my generator, if I do need additional files like the additional text or if I do need the options, I can keep combining them until I have this one big provider that I then register my output with. And in this, which is the output we see, we have here tuple with multiple left and rights. And if I jump into this callback, we actually see this tuple being fed in. And from here, we can now do our generation logic. So in our case, we find out do is this external init keyword already defined in the compilation? What’s the C# language version? And so on and so forth.
Stefan (01:09:59)
And then eventually, again from a generation point of view, it’s basically the same. We need some string that we can then put out into our compilation and actually end up with this IsExternalInit type emitted if we need it. And this is now the different programming model of incremental generators. So this is a little bit less straightforward, but eliminates potential performance issues. Basically mentioned that. Yeah, so this is the consideration about performance. Our source generator, they’re not async. So, they’re all sync because while we are waiting for this compilation so that we can continue editing in Visual Studio and Rider. So if we’re doing something more expensive or some longer loop, then regularly check if the cancellation has been requested already because it doesn’t make any sense to keep generating something which will be discarded anyway or is already basically outdated. And we discussed the issue of the syntax receiver in huge project or a potential issue.
Stefan (01:11:20)
Talking about NuGet. Now, if you want to pack this analyzer and ship it on NuGet, I briefly mentioned it already, what we want to do is to always set a development dependency to true because this will then per default… If we .NET install this package, then per default, we get the private assets all which you always want because analyzers per default shouldn’t flow to tentative dependencies. And since we did multi-targeting, our analyzer targets both .NET 6 and .NET Standard 2.0. We need to select the correct target framework which is Standard 2.0. Analyzer must be .NET Standard 2.0. So, this is what we can do with this line. We indeed select the NET standard 2.0 target and pack it into this directory. This is where the analyzer lives in the NuGet package. We saw that already.
Stefan (01:12:16)
And for the end, I would like to highlight… Actually, almost the end. I want to highlight a couple of examples. I’ll link this all in the presentation. For example, there is a C# source generator repository where the community has collected a lot of generators. So maybe, there is one already in there which solves an issue of yours. Recently also, the community toolkit has been released which has a source generator for… If I find it. Component only, yes. For the I notify property changed event. So I notify property changed is usually something that is tedious implementation. It always follows a very similar pattern. So, we can have a source generator generating this for us. The user of this generator has less code to manage and to maintain.
Stefan (01:13:26)
I mentioned testing. With this testing package, we can very nicely use… We can debug analyzes. We can test our… Sorry, generators. We can test our generators. We can also do code coverage on our generators and also more advanced scenarios such as mutation testing. So, I’m using Stryker.NET here for mutation testing. And I want to show the report here. So, let’s have a look at the generator that we inspected, the EnumInfo generator. And we basically run the command line tool .NET Stryker against it. And since we have basically our X unit or end unit or MSTest Suite, this works just perfectly. If you’re not familiar with mutation testing, there is quite a lot of video tests on the web. But if you are, we see we have killed a lot of mutants. A couple of them have survived. I seem to not exclude my debug asserts here and see what else has survived here. Here, these seem if we… Let’s see what mutation testing does. So mutation testing changes this greater than or equal to, to just a greater than and then no test fails.
Stefan (01:14:51)
I definitely need to add a couple of tests to this very project. You can also do benchmarking both on generators and also the generated code. With the generator itself, it’s a little bit more tricky because the driver that we use does internally some caching. So we can’t get, for example if benchmark.net, not the most precise reports. But we could get an indicator where we basically… Have a look at an example. Benchmark where we again feed in some source to this driver, and then basically run our benchmark against where we invoked it. I just briefly step over it. Somewhere is a generator driver that we use in order to run the generator and update the compilation. And we can also benchmark the generated code.
Stefan (01:15:58)
I’m afraid I might have deleted the report, but basically I here have a bit of a benchmark where I compare the generated version of this EnumInfo.GetName, but also the regular Enum.ToString and the already existing Enum.GetName. And here, you would then see that the generated version, if I step into it, this is our generated version, is measurably faster and also allocates less than… Actually, it allocates nothing. So, this is also allocation free. And the Enum ToString or the Enum GetName actually allocates a little bit. And yeah, if you would like to know more about source generators, in this presentation I’ll link here a couple of videos. For example, a recent video here is about how incremental short generators work under the hood and how this pipeline is internally managed, that this data internally managed and updated or reused.
Stefan (01:17:10)
And yeah, before we hand over to the Q and A section, Gael is about to show something very exciting. I just want to have one more note. So if you have, later on after the Q and A, some questions, then you may contact me on Twitter. I am also now twitch streaming on Twitch.Flashover. And actually, in two weeks on Saturday, I will be doing a live stream on source generators where we will build live source generator. All of the source is available here. I will put it into the webinar chat if you want to check it out later.
Gael (01:17:53)
Thank you, Stephan. It was amazing, very detailed. I learned a lot even if I’ve been working with source generators.
Gael (01:18:00)
I’ve been working with source generators for a couple of months and didn’t knew everything. I would like to show a product that, actually, we’ve just released a preview of this week. Let me share my screen. I will do a very short introduction to this.
Gael (01:19:05)
So, what you have shown, Stefan, this is actually, from my point of view, very, very low level development, because you need to cope with the syntax nodes, with the semantic model. And, I think, that what Microsoft did with that, is a low level API to extend Roslyn. And, there are many of these APIs. You mentioned analyzers, there are also suppressors, there are code fixes and code refactorings. And, we have built a product evolved from PostSharp, or the same kind of project as PostSharp, for high level metaprogramming. So, you can do all these things, but without going to the syntax node.
Gael (01:20:07)
And, the concept of Metalama, is that we have a templating language that we call T#. T#, like templating for C#. And, this is an example of a template that does login. So, this is the template. You see, what is great here, is compile time. So this is a compile time expression. So, we are entering methods. Method that proceeds here, is the call to the method that we are templating. And then, after that, we are writing another message. So, I can apply the template to any method, and I can preview what the template does to my code. So, you see here, we are doing code generation, but there are big differences with the Roslyn code generators, because we are able to modify a method itself, not just a partial [inaudible 01:21:18], so it is not additive only, it can also add things.
Gael (01:21:22)
So, this is one example. Another simple example of a template you may do is, for instance, we are invoking the method inside the loop. We have also a catch and win, and we can still retry. We are just sleeping and looping. So, if we apply this retry to a method, and I’m doing the preview, we can see what Metalama will do with the code. So, the difference is that here, it would do this with the code at compile time, because at run time it doesn’t want to edit your code. There would be no point.
Gael (01:22:17)
And, you can also do more complex templates. So, here I would like to do login with parameters. So, instead of just saying, “I’m starting the method,” I will like to have the method name with the parameters, and I could add another parameter and see what it is doing.
Gael (01:22:43)
So, I can do the diff, and this aspect, or this template… We can call that an aspect… is printing the name of each parameter. So, it’s actually just a more complete, more complex template, where we are creating an interpolated StringBuilder. We are going through the parameters, we are adding text tokens. And, you see, this is C#. It is not Roslyn programming. It is C#, it’s the same level of abstraction of programming as you are using, when you are doing uWISE, who has business applications. You don’t need to go into the details of Roslyn. And, we can also do what source generators can do, like introducing new things. Here, this template, that we can apply to a type, we are introducing a property into that type, and I can do the preview to see what’s happening. So, the good thing, is that we can combine.
Gael (01:23:56)
So, you have shown the implementation of notify property change through source generators. I, frankly, believe this is a shame, not of you, Stefan, but of Microsoft, to show notify property change as an example of source generators. Because, what they you do, is that they’re transforming fields into properties. So, generating fields from properties. So, actually, you are losing the idiomatic C#. It’s no longer a C#, what you are doing is just hacking. So, we know how to do notify property change. We’ve done that for 10 years. You don’t want to change your properties into fields, just to have your source generators to work.
Gael (01:24:48)
So, here we have a more complex aspect that, actually, combines two things. We are introducing a method, we are introducing a property, we are implementing the interface and, at the same time, we are changing all property setters to detect a change in the value. This is the right way to do that, because your existing code doesn’t need to be changed. Source generators can be abused, they have been abused. You should not do that. That’s a matter of opinion, that we could discuss.
Gael (01:25:20)
So now, if you apply notify property change to an existing code, actually, what it’s going to do, is to change your code, to inject the logic. It applies the templates. And, you can notice that we can, actually, call from the source code, the code that has been implemented. Something that was not possible with PostSharp, because we did that at post compile time. And, this is because we are able to do that, because we integrate with source generators. So, here you can see that we have our source generators, which defines the new methods. You will see these new methods are not implemented. I believe there is no point at all in generating at design time, the bodies of methods. It just takes CTU time. It is useless. It’s only useful when you compile that. So, we don’t do that, we only define the declarations, not the body.
Gael (01:26:31)
Now, if you choose here, LAMA debug configuration. You can also debug the source that we have produced. So, you can debug the generated code, the transformer code. Now you get into the property settor. So, we have that.
Gael (01:26:52)
And, that’s all I wanted to show today, because there is a lot more. You can also emit warnings, suppress warnings. I’ve shown that you can do code fixes and code refactorings using templates. I didn’t show that. And, I have also shown the code transformers. That’s all I wanted to show today, because that’s not a webinar about Metalama. But, if you are interested, there are two things you can do. So, if you are interested to try it today, you can go to our website, postsharp.net, into the Metalama section. There is also a sandbox, an online sandbox, actually a fork of Try.NET. So, this is a fork, where you can try Metalama without leaving your browser. The next webinar, on March 15th, will be about Metalama. The exact time is not decided yet, but we will perhaps do two times in the day. One in the European afternoon, the second in the European evening. That’s all I have for MetaLama.
Questions
Gael (01:28:22)
And, let’s see if there are questions? To see questions, I may need to stop sharing. And, I don’t see questions. But, if there is no question, I have one question, actually. You have shown mutation test. Could you elaborate? It’s the first time I heard about mutation test, and I would like to hear, what is it I’m missing?
Stefan (01:28:58)
Perhaps, let me briefly dig out the example that I have. There’s there was, actually, a mutation testing talk that I also gave at the NTC Oslo, so let me fire up that solution. I like to show it by example.
Gael (01:29:23)
You need to share a screen again.
Stefan (01:29:24)
Yes, yes, yes. I’m on it. Oops, don’t want to cause any recursion. So, in this example, let’s actually run all the tests and see if everything indeed turn green.
Stefan (01:29:47)
And, I have a bit of a… Oops. Briefly get rid of that by terminating global chase number. I believe I’m pinning to an older, it’s the key version, which I no longer have. And, for example, I have here a naive level calculator where I can add numbers, I subtract numbers, I can multiply, and so on and so forth. And, actually, I do have a little bug in here. I do have the divide method. I have here, by accident, also a multiplication, so I know I did a bug here. But, what troubles me now, that all my tests are green. And, actually, if I would do a coverage, a code coverage, report on that, I would see that I have 100% code coverage. I got everything covered. It’s actually jumped to the test that we have here, so I call this, “Divide.” I divide one by one and expect one. And, I divide 240 by zero, and then expect this NIN that I defined, [inaudible 01:31:03] exception.
Stefan (01:31:04)
So, I have 100% code coverage, but I did not discover the bug here, because my test suite isn’t good enough, and this is what mutation testing now uncovers. So, what mutation testing does, it has a look at tests, sees what is the project under test? And then, from this project under test, actually mutates the source code. So, it goes ahead and changes, for example, this addition to a negation. So, it created one mutant. Now, with this modified source, now all the tests are one again. Or, at least, all impacted tests, which is tests which actually now cover this production line.
Stefan (01:31:55)
And now, we expect at least one test to fail, and it does. So, we see here, at least one test is failing, and this is good. So, we have produced the mutant, but now one test is failing. That’s good. So, the mutant has been killed. So, we not only cover… Oh, where is it? So, we only cover this line of code, we also have an assertion that protects this line of code, or this very statement. And then, rotation testing continues and changes again, for example, a minus to a plus, and so on and so forth. And, eventually, down here, and I would change this multiplication, for example, to a plus, something else. And now, I got… Sorry, actually, it changes, in that case, to a division. Oops, I hit [inaudible 01:32:53]. And, we see now, we have no failing test. Although we changed this, basically we fixed it. But, we have no failing test, which means that this line isn’t… Well, it is covered, but it’s not 100% asserted. So, changing the code, won’t make a test to fail. So, I could now add a better test. Let’s say, I divide four by two and expect two. And, I restore the original code, and now I have improved my test suite. And now, my test suite fails, and now I really discovered that bug.
Stefan (01:33:40)
So, mutation testing, in short, it creates mutants of the production code and each mutant expects at least one test to fail. And, if a test is failing, the mutant is killed. That’s excellent. And, if no test fails, then the mutant survives. And, we then, at the end, at the mutation coverage report… Do we still have it open from previously? No, I’m afraid I don’t. But, in this mutation test report, we then can see, “Okay, what has been mutated? Where is our test suite not 100% effective yet, so we can improve our tests and eventually then fix more bugs?”
Gael (01:34:19)
Okay. We have a question from the audience, “Do you know if the .NET Standard 2 restriction for source generators will be changed in the future, so we can use later target frameworks like .NET Standard 2.1 or higher?”
Stefan (01:34:37)
I’m afraid, I don’t know that. I am not sure. I believe, but this is my understanding of it, that as long as Visual Studio is working as it is, and is on .NET Framework, then the .NET Standard restriction will keep alive, for backwards compatibility. If there are any plans to change it in the future, maybe there is a different solution around it, but I’m not aware of that, I’m afraid.
Gael (01:35:11)
Yes. I agree. I-
Stefan (01:35:12)
This not only impacts source generators, this impacts Roslyn, plugins, like code fixers, analyzers, in general.
Gael (01:35:22)
Yeah. We had the same discussion internally, and it needs to stay .NET Standard 2. 0. The way we solve that in Metalama, is that only the compile time code needs to be to .NET Standard 2.0, because actually, when we compile your project, we split it between run time and compile time. But, anything that is compile time, it must be .NET Standard 2.0. I was disappointed to see that visual studio 2022 was on the .NET Framework run time and not on the . NET 6 run time. That would have solved a lot of problems, but I think it is not so easy.
Gael (01:36:07)
We have another question. “If we don’t emit the generator to disk, which is recommended, I believe, does this affect the debugging experience, if I want to step into the generator code from this project?”
Stefan (01:36:23)
If I understand correctly, actually it’s the other round. So, for protection projects, it’s not recommended to really emit the files to disk. Those are just on the [inaudible 01:36:36] for debugging, because it will take additional resources to put this file to disk, so we need to enable that. Per default, they are not emitted to disk, so we need to explicitly enable this for this debugging experience, that we can view, for example. What I like to do, is then have, in a second screen, the generated file in, while I edit the source generator. So, per default, it’s off. For debugging, we need to enable it. But, I don’t recommend to enable it for the production project itself, I like to enable it for a small testing project there. I always have a sample project, rather than actually integration test the generator.
Gael (01:37:27)
So, by default, the files are in the PDB. So, the generated files are in the PDB. I’m not sure if it’s in all PDBs or just in the new PDB format. Do you know anything about that?
Stefan (01:37:40)
Oh, I’m afraid, I don’t. No.
Gael (01:37:44)
And so, pay attention. Well, you should not ship your PDB, if your source code is confidential. But, with source generators, your PDB really contains source code, but only the generated one.
Stefan (01:38:01)
Which, is also now an interesting thing, to be careful about which… Well, this applies for every Newgate package, but from which source we consume source generators? Well, we could depend on a malicious package, which has malicious code, which maybe read my local files and puts them to a server. But, we could also, with a source generator, now a tech could have, basically, access to our source code, to a source tree. So, from a security point of view, this is something to consider, safe sources for our generators, because they could be, technically, abused for an attack, or for malicious evilness.
Gael (01:38:52)
You mean, because the generator or the analyzer runs inside the Visual Studio, [crosstalk 01:39:04]
Stefan (01:39:04)
Not specific in Visual Studio, but in general.
Gael (01:39:07)
Yes.
Stefan (01:39:08)
Since the generator sees the source. Now, the tooling could put it out to file for us to debug, but an attacker, a malicious package, could put this to some server and could leak our source code, yeah.
Gael (01:39:24)
But, this was there from the beginning of Newgate. You could have a MSBuild task that was a DLL, and then you got access to the local machine.
Stefan (01:39:36)
Oh, okay.
Gael (01:39:37)
Yeah.
Stefan (01:39:37)
I wasn’t aware of that.
Gael (01:39:41)
In the beginning of Newgate, it was a problem. So, actually, companies didn’t allow… Well, not all companies, of course, but corporates didn’t allow to consume packages from newgate.org, but they had their own repositories of curated repositories. But now, there are decent security settings on Newgate and you can define trusted publishers. So, for instance, we are signing all our packages. So, if you trust us, you can add our signature to your list of trusted publishers. But, yes, the risk, it was always there. If you consume a Newgate package, you just open the doors of your computer to the author.
Gael (01:40:33)
Good. We have run way over time, but it was very interesting. And, I guess, people who were not interested, left already, so we could continue. But anyway, I don’t see more questions from the audience. So, Stefan, thank you very much for this evening, for this presentation. It was very interesting. And, next month there will be a more detailed presentation about Metalama, based on source generators and analyzers, and all these technologies. No longer on MS [inaudible 01:41:12]. And, if you want more details faster, you can go to our website today. Thank you very much. I’m going to stop the recording and stop the webinar in a minute. Thank you, and bye-bye.
Stefan (01:41:27)
Thank very you much. Thank you for having me.
Gael (01:41:28)
Bye.
Stefan (01:41:29)
I had a lot of fun, and I’m looking forward to the Metalama presentation.
Gael (01:41:33)
Thank you. Bye-bye.
Stefan (01:41:34)
Bye-bye.
