.NET MAUI

.NET MAUI – Exploring Overlays – Part 2

This is post #4 in a series called ‘.NET MAUI Source of Truth’.

About Source Of Truth – As any developer knows, source code is the purest form of truth in working software. So I’ve decided the best way to get deep into .NET MAUI is to look at the source code.

In my last posts, we explored the .NET MAUI codebase learning about the new Windows functionality, and then managed to get a demo of them working in Preview11. Then we’ve dived into overlays in .NET MAUI.

After writing my previous post on Overlays I found an interesting piece of code in the .NET MAUI codebase. This code was basically a advanced implementation of a WindowsOverlay. This was called the VisualDiagnosticsOverlay and I also managed to find a good sample project called DrasticOverlay created by https://twitter.com/drasticactionSA.

This sample project is a good example of what can be done with overlays. This sample project has example of Overlays, Background Overlays, Hit Detection Overlays, Videos, Menus, Loading, Drag and Drop and more.

Looking through this codebase we can see the different overlays that have been created, see below.

In this case, let’s dig into the drag and drop overlay to see how these awesome overlays are working. The basic implementation of an overlay is a partial class that inherits from WindowOverlay. The partial class combined with multi-targeting allows for common code and native implementations to nicely sit side by side.

Below, I’ve included the DragAndDropOverlay code. In this, we can see our partial class and inheritance from WindowOverlay. Overall it’s a fairly simple class that has a drop element, which is a IWindowOverlayElement. In this case the element is using the drawing capabilities available in .NET MAUI. Then a simple event handler and IsDragging property.

public partial class DragAndDropOverlay : WindowOverlay
{
    DropElementOverlay dropElement;
    bool dragAndDropOverlayNativeElementsInitialized;

    internal bool IsDragging
    {
        get => dropElement.IsDragging;
        set
        {
            dropElement.IsDragging = value;
            this.Invalidate();
        }
    }

    public DragAndDropOverlay(IWindow window)
        : base(window)
    {
        this.dropElement = new DropElementOverlay();
        this.AddWindowElement(dropElement);
    }

    public event EventHandler<DragAndDropOverlayTappedEventArgs>? Drop;

    class DropElementOverlay : IWindowOverlayElement
    {
        public bool IsDragging { get; set; }
        // We are not going to use Contains for this.
        // We're gonna set if it's invoked externally.
        public bool Contains(Point point) => false;

        public void Draw(ICanvas canvas, RectangleF dirtyRect)
        {
            if (!this.IsDragging)
                return;

            // We're going to fill the screen with a transparent
            // color to show the drag and drop is happening.
            canvas.FillColor = Color.FromRgba(225, 0, 0, 100);
            canvas.FillRectangle(dirtyRect);
        }
    }
}

As we can see in the previous image there are native implementations on Windows, iOS and Android. Let’s now take a look at the iOS partial class, this class is named DragAndDropOverlay.iOS.cs. You can see the full file over here:

Below is the Initialize() method on the iOS implementation. There’s a native view called DragAndDropView and this is added as a subview on the nativeWindow?.

// We're going to create a new view.
// This will handle the "drop" events, and nothing else.
dragAndDropView = new DragAndDropView(this, nativeWindow.RootViewController.View.Frame);
dragAndDropView.UserInteractionEnabled = true;
nativeWindow?.RootViewController.View.AddSubview(dragAndDropView);
nativeWindow?.RootViewController.View.BringSubviewToFront(dragAndDropView);

Below we can see part of the DragAndDropView which is implemented in the iOS code. We can see that it’s making use of native APIs eg UIView and IUIDropInteractionDelegate.

class DragAndDropView : UIView, IUIDropInteractionDelegate
{
    DragAndDropOverlay overlay;

    public DragAndDropView(DragAndDropOverlay overlay, CGRect frame)
        : base(frame)
    {
        this.overlay = overlay;
        this.AddInteraction(new UIDropInteraction(this));
    }

    [Export("dropInteraction:canHandleSession:")]
    public bool CanHandleSession(UIDropInteraction interaction, IUIDropSession session)
    {
        Console.WriteLine($"CanHandleSession ({interaction}, {session})");

        return session.CanLoadObjects(new Class(typeof(UIImage)));
    }
    ...
}

In looking at the majority of the implementations they all follow a similar pattern to the DragAndDrop Overlay. As mentioned previously you can implement Overlay on multiple levels, so you can use the Cross-Platform drawing methods available in .NET MAUI and you can also do native implementations for each platform.

If you are interested in overlays I would recommend taking a look at this cool sample project – DrasticOverlay and the VisualDiagnosticOverlay from the .NET MAUI codebase.

Overall it’s nice to see that we can implement this level of functionality within overlays in .NET MAUI. Ideally I would like to see support for .NET MAUI controls inside overlays, presently I’m not sure how we could implement this but I suspect it would be possible.

.NET MAUI – Exploring Overlays

This is post #3 in a series called ‘.NET MAUI Source of Truth’.

About Source Of Truth – As any developer knows, source code is the purest form of truth in working software. So I’ve decided the best way to get deep into .NET MAUI is to look at the source code.

In my last posts, we explored the .NET MAUI codebase learning about the new Windows functionality, and then managed to get a demo of them working in Preview11.

In my previous research, I noticed a new method on Window called AddOverlay. In this post I’m going to explore WindowOverlays, what are they? How is it implemented natively? How does it maintain state, including during page navigation? When should I use them? How is it related to the traditional navigation style and Shell? What can we use it for? (login screens/flows) How do we animate it?

Going back to the roots of ‘Source of Truth’ this post is going to focus on the .NET MAUI source code and look at how things are implemented.

To do this post I’ve downloaded the .NET MAUI source code from github and opened it in Visual Studio for Mac Preview.

If we start at the source we can see that IWindow has 2 methods and a property in relation to Overlays.

/// <summary>
/// Provides the ability to create, configure, show, and manage Windows.
/// </summary>
public interface IWindow : ITitledElement
{
	/// <summary>
	/// Gets the read only collection of Window Overlays on top of the Window.
	/// </summary>
	IReadOnlyCollection<IWindowOverlay> Overlays { get; }

	/// <summary>
	/// Adds a Window Overlay to the current Window.
	/// </summary>
	/// <param name="overlay"><see cref="IWindowOverlay"/>.</param>
	/// <returns>Boolean if the window overlay was added.</returns>
	bool AddOverlay(IWindowOverlay overlay);

	/// <summary>
	/// Removes a Window Overlay to the current Window.
	/// </summary>
	/// <param name="overlay"><see cref="IWindowOverlay"/>.</param>
	/// <returns>Boolean if the window overlay was removed.</returns>
	bool RemoveOverlay(IWindowOverlay overlay);

As my next step I thought I would take a look in the codebase looking to see where AddOverlay is used, (hoping we had a sample). In the search, I found a test and a sample. Using the overlay methods seems simple enough.

void TestAddOverlayWindow(object sender, EventArgs e)
{
	var window = GetParentWindow();
	overlay ??= new TestWindowOverlay(window);
	window.AddOverlay(overlay);
}

void TestRemoveOverlayWindow(object sender, EventArgs e)
{
	if (overlay is not null)
	{
		GetParentWindow().RemoveOverlay(overlay);
		overlay = null;
	}
}

Next, I’ve taken a look at how the AddOverlay method is implemented in the Window(.NET MAUI Window), Window.Impl.cs.

/// <inheritdoc/>
public bool AddOverlay(IWindowOverlay overlay)
{
	if (overlay is IVisualDiagnosticsOverlay)
		return false;

	// Add the overlay. If it's added, 
	// Initalize the native layer if it wasn't already,
	// and call invalidate so it will be drawn.
	var result = _overlays.Add(overlay);
	if (result)
	{
		overlay.Initialize();
		overlay.Invalidate();
	}

	return result;
}

It’s a simple method, the overlay gets added to the list, then initialised and invalidated. In this method, it doesn’t show how these overlays are rendered onto the screen. In order to find this out, we will need to do a bit more digging.

At this point in time, you might ask the question has this functionality even been implemented? I had the same question so I’ve taken the sample code from my previous blog, you can find here, and added some code to test. I added and overlayed the following:

public class TestWindowOverlay : WindowOverlay
{
	IWindowOverlayElement _testWindowDrawable;

	public TestWindowOverlay(Window window)
		: base(window)
	{
		_testWindowDrawable = new TestOverlayElement(this);

		AddWindowElement(_testWindowDrawable);

		EnableDrawableTouchHandling = true;
		Tapped += OnTapped;
	}

	async void OnTapped(object sender, WindowOverlayTappedEventArgs e)
	{
		if (!e.WindowOverlayElements.Contains(_testWindowDrawable))
			return;

		var window = Application.Current.Windows.FirstOrDefault(w => w == Window);

		System.Diagnostics.Debug.WriteLine($"Tapped the test overlay button.");

		var result = await window.Page.DisplayActionSheet(
			"Greetings from Visual Studio Client Experiences!",
			"Goodbye!",
			null,
			"Do something", "Do something else", "Do something... with feeling.");

		System.Diagnostics.Debug.WriteLine(result);
	}

	class TestOverlayElement : IWindowOverlayElement
	{
		readonly WindowOverlay _overlay;
		Circle _circle = new Circle(0, 0, 0);

		public TestOverlayElement(WindowOverlay overlay)
		{
			_overlay = overlay;
		}

		public void Draw(ICanvas canvas, RectangleF dirtyRect)
		{
			canvas.FillColor = Color.FromRgba(255, 0, 0, 225);
			canvas.StrokeColor = Color.FromRgba(225, 0, 0, 225);
			canvas.FontColor = Colors.Orange;
			canvas.FontSize = 40f;

			var centerX = dirtyRect.Width - 50;
			var centerY = dirtyRect.Height - 50;
			_circle = new Circle(centerX, centerY, 40);

			canvas.FillCircle(centerX, centerY, 40);
			canvas.DrawString("🔥", centerX, centerY + 10, HorizontalAlignment.Center);
		}

		public bool Contains(Point point) =>
			_circle.ContainsPoint(new Point(point.X / _overlay.Density, point.Y / _overlay.Density));

		struct Circle
		{
			public float Radius;
			public PointF Center;

			public Circle(float x, float y, float r)
			{
				Radius = r;
				Center = new PointF(x, y);
			}

			public bool ContainsPoint(Point p) =>
				p.X <= Center.X + Radius &&
				p.X >= Center.X - Radius &&
				p.Y <= Center.Y + Radius &&
				p.Y >= Center.Y - Radius;
		}
	}
}

On my iPad simulator, I was able to see the drawing. The tapped method was not working for me.

Now we know the drawing code is working but not how it is all linked together, the clues might just be IWindowOverlayElement and Draw.

If we then take a look in the MAUI codebase we can see that the Draw method also exists on the WindowOverlay, it loops over the elements and draws them. I assume (hope) there’s some type of visual invalidation hierarchy in this window overlaying.

/// <inheritdoc/>
public void Draw(ICanvas canvas, RectangleF dirtyRect)
{
	if (!IsVisible)
		return;
	foreach (var drawable in _windowElements)
		drawable.Draw(canvas, dirtyRect);
}

Also noting that I’ve discovered WindowsOverlay is a partial class and there’s a native implementation in each.

Now I need to find out how/when/why the Draw method is called.

After a bit of digging, I found some answers inside the native implementations of the WindowOverlay. What happens is that the WindowOverlay grabs the native Layer and then draws itself on top using the NativeGraphicsView to map the drawing in the Overlay to a view that can be seen natively. Below is the iOS version.

// Create a passthrough view for holding the canvas and other diagnostics tools.
_passthroughView = new PassthroughView(this, nativeWindow.RootViewController.View.Frame);

// the native graphics view calls the draw methods
_graphicsView = new NativeGraphicsView(_passthroughView.Frame, this, new DirectRenderer());
_graphicsView.AutoresizingMask = UIViewAutoresizing.All;

_passthroughView.AddSubview(_graphicsView);

...

// Any time the frame gets a new value, we need to update and invalidate the canvas.
_frameObserver = nativeLayer.AddObserver("frame", Foundation.NSKeyValueObservingOptions.New, FrameAction);
// Disable the graphics view from user input.
// This will be handled by the passthrough view.
_graphicsView.UserInteractionEnabled = false;

// Make the canvas view transparent.
_graphicsView.BackgroundColor = UIColor.FromWhiteAlpha(1, 0.0f);

// Add the passthrough view to the front of the stack.
nativeWindow.RootViewController.View.AddSubview(_passthroughView);
nativeWindow.RootViewController.View.BringSubviewToFront(_passthroughView);

// Any time the passthrough view is touched, handle it.
_passthroughView.OnTouch += UIViewOnTouch;
IsNativeViewInitialized = true;
return IsNativeViewInitialized;

It looks like the only way to use overlays is through canvas drawing, this means none of the .NET MAUI Controls or functionality will work with overlays. Some initial problems that I can see at this point, what about transitions and animations?.

As a basic requirement we need to update the UI based on state changes, I was able to do this by using the Invalidate method on the Overlay.

Device.StartTimer(TimeSpan.FromMilliseconds(500), () =>
{
	_size += 10;
	_overlay.Invalidate();
	return true;
});

So we’ve coved the basic questions, what are they and how is it implemented natively. What about the other questions?

How does it maintain state, including during page navigation? I have not tested this specifically but based on the architecture I would suggest it does maintain the state.

Do gestures pass-through to underlying views? This worked well, when I tapped where the drawing was then gestures did not pass-through but when I tapped outside of the drawing the gestures did pass-through.

How is it related to the traditional navigation style and Shell? This functionality sits completely outside of navigation and shell.

What can we use it for? In the future I would like to see it used to build outstanding user experiences, eg popovers, toast messages, alerts and slideovers.

How do we animate/transition it? Not sure exactly, I don’t think we can use any of the the animation libraries available in .NET MAUI (.NET MAUI Animations, Lottie or Native APIs).

Summary

  • Uses the Drawing capability of .NET MAUI
  • We can have multiple overlays and multiple overlay elements.
  • We can add and remove overlays
  • Overlays provide a canvas and draw method which allows you to draw on the screen.
  • No support for Controls, Animations or Transitions
  • Has the ability to invalidate and redraw

My thoughts

I think that overlays are a great idea and something that I could get excited about but in my current knowledge it seems that overlays provide a fairly limited functionality. Considering that the overlay feature needs to be supported and maintained if built, then ideally it should be a useful feature.

I think that with overlays you should be able to create something like slideoverkit and popups with animations/transitions etc, ideally it would support all the functionality create these easily or allow the extensibilty to create these.

I would suggest it will need to be able to support controls, animations and transitions.

We know that .NET MAUI is still early days so I hope that we can see this functionality developed further.

Exploring Multi-window Apps in .NET MAUI Preview 11

This is post #2 in a series called ‘.NET MAUI Source of Truth’.

About Source Of Truth – As any developer knows, source code is the purest form of truth in working software. So I’ve decided the best way to get deep into .NET MAUI is to look at the source code.

In my last post we explored the .NET MAUI codebase learning about the new Windows functionality but we could not get experience with overlays until Preview11. The great news is that Preview11 has been shipped.

You can learn more about the preview here: https://devblogs.microsoft.com/dotnet/announcing-dotnet-maui-preview-11/

I’m doing all this on the mac. You can also do this on Windows but you’ll need a ipad. If you want to use your Mac you can, you can see some details on installing this on a mac, you can find more detailed installation and upgrade notes at these locations:
https://github.com/dotnet/maui/wiki/macOS-Install
https://xam.com.au/installing-net-maui-preview/

On another note the documentation for .NET doesn’t provide instructions to set your path for dotnet permanently. If you want to do this then following these instructions. I found dotnet located here: /usr/local/share/dotnet/dotnet. 

In my case I already had dotnet and maui installed, so I just needed to do an upgrade. At the time that I wrote this post then Maui Check was not upgrading me to Preview 11, but do note I did first run Maui Check and it resolved a few other issues for me so I would recommend doing a check first.

Here’s what I did.

1. Run Maui Check.

cd $HOME/.dotnet/tools
./maui-check

Then I resolve resolved all the issues associated

2. Manually update to preview11

sudo dotnet workload install maui
dotnet new --install Microsoft.Maui.Templates
dotnet new maui -n MauiPreview11Play
cd MauiPreview1Play
dotnet restore
dotnet build -t:Run -f net6.0-ios

Generally in .NET MAUI development(on the mac) I’ve been able to use Visual Studio Mac Preview, in this case I’ve updated to the latest version and now I can open the project.

Update: Initially I started with using VS MAC Preview but eventually it caused too many issues, not that I think this was a VS issue but more that .NET MAUI was not building and deploying without issues. Eventually I turned to VS Code the command line.

It took me a long time to get this working because I would have issues building and deploying, I was not able to tell if it was my issues or .NET MAUI. I had to switch between –no-incremental and a normal build when I had issues with build and deploy.

dotnet build --no-incremental -t:Run -f net6.0-maccatalyst / ios

dotnet build -t:Run -f net6.0-maccatalyst / ios

Multi-Window


Once we have the new project running we can start our setup for Multi-window.

Step 1. Add a SceneDelegate, you can add this under Platforms/MacCatalyst and Platforms/iOS.

using Foundation;
using Microsoft.Maui;
using ObjCRuntime;
using UIKit;

namespace MauiPreview11Play;

[Register("SceneDelegate")]
public class SceneDelegate : MauiUISceneDelegate
{

}

Step 2. Update your info.plist to support multiple scenes. You can do this under /Platforms/iOS and /Platforms/MacCatalyst

	<key>UIApplicationSceneManifest</key>
	<dict>
		<key>UIApplicationSupportsMultipleScenes</key>
		<true/>
		<key>UISceneConfigurations</key>
		<dict>
			<key>UIWindowSceneSessionRoleApplication</key>
			<array>
				<dict>
					<key>UISceneConfigurationName</key>
					<string>__MAUI_DEFAULT_SCENE_CONFIGURATION__</string>
					<key>UISceneDelegateClassName</key>
					<string>SceneDelegate</string>
				</dict>
			</array>
		</dict>
	</dict>
	<key>NSUserActivityTypes</key>
	<array>
		<string>com.companyname.mauipreview11play</string>
	</array>


Step 3. Setup the multi-window code. In this case I’ve just edited the existing files to add some buttons and methods.





<Button 
    Text="Open window"
    FontAttributes="Bold"
    Grid.Row="4"
    SemanticProperties.Hint="open it"
    Clicked="OpenWindow"
    HorizontalOptions="Center" />

<Button 
    Text="Close window"
    FontAttributes="Bold"
    Grid.Row="5"
    SemanticProperties.Hint="Close it"
    Clicked="CloseWindow"
    HorizontalOptions="Center" />


private void OpenWindow(object sender, EventArgs e)
{
	Application.Current.OpenWindow(new Window(new MainPage()))
}

In order to open a window we can use the OpenWindow method on the Application, providing a new Window.


private void CloseWindow(object sender, EventArgs e)
{
	var window = this.GetParentWindow();
	if (window is not null)
		Application.Current.CloseWindow(window);
}

In order to close a window we need to provide the window.

Here’s a video of multiple windows in a maccatalst app

Also multiwindow in iPad.

There we go, .NET MAUI Multi-Windows Apps. If you want to see the code you can find it here: https://github.com/rid00z/NETMAUIMultiWindow

Thanks

.NET MAUI Source Of Truth – Exploring IWindow

This is post #1 in a series called ‘.NET MAUI Source of Truth’.

About Source Of Truth – As any developer knows, source code is the purest form of truth in working software. So I’ve decided the best way to get deep into .NET MAUI is to look at the source code.

Exploring IWindow

Recently I was exploring the .NET MAUI codebase and came across something that seemed interesting, IWindow. ‘Windows in MAUI? that doesn’t make sense’ because Xamarin.Forms never had windows. I set about to find out what IWindow was.

A Xamarin.Forms Recap

Before we take a look at Windows in .NET MAUI let’s refresh ourselves on view hierarchies in Xamarin.Forms. In Xamarin.Forms we have an Application class with a MainPage property which takes a Page. You’re able to set the MainPage to either a single page or a navigation page.

So our hierarchies looks something like this:
Application->Page(Navigation/Page)->Page->Content

Windows in MAUI

Important Note: In this article whenever I make reference to Window/Windows most of the time it’s going to be Windows concept from .NET MAUI, not the Windows platform support of .NET MAUI.

If I look into any samples of MAUI applications then they follow the same hierarchy we find in Xamarin.Forms. So where’s the Windows?

Initially looking at the latest preview10 of .NET MAUI it seems the functionality of Windows is limited. So in order to see what Windows are going to look like in the future of .NET MAUI then we need to look at the latest source code.

If we remember that we’ve always started in Forms applications using MainPage,

eg MainPage = new ContentPage();

then let’s start with that MainPage property on the application class, FYI this is code from the MAUI github repository. If we dig into that method we can see that the MainPage still is part of the hierarchy but now Window is a parent of that page.

public Page? MainPage
{
	get
	{
		...
	}
	set
	{
		if (MainPage == value)
			return;

		OnPropertyChanging();

		if (Windows.Count == 0)
		{
			_pendingMainPage = value;
		}
		else
		{
			Windows[0].Page = value;
		}

		OnPropertyChanged();
	}
}

Now we can see that Window is a parent of Page, and Window has a property called Page.

If we look further into the implementation of the application class then we can see a new method that creates a new Window. I guess with the variable _pendingMainPage that we require some type of lazy loading.

IWindow IApplication.CreateWindow(IActivationState? activationState)
{
	Window? window = null;

	// try get the window that is pending
	if (activationState?.State?.TryGetValue(MauiWindowIdKey, out var requestedWindowId) ?? false)
	{
		if (requestedWindowId != null && _requestedWindows.TryGetValue(requestedWindowId, out var w))
			window = w;
	}

	// create a new one if there is no pending windows
	if (window == null)
	{
		window = CreateWindow(activationState);

		if (_pendingMainPage != null && window.Page != null && window.Page != _pendingMainPage)
			throw new InvalidOperationException($"Both {nameof(MainPage)} was set and {nameof(Application.CreateWindow)} was overridden to provide a page.");

		// clear out the pending main page as this will never be used again
		_pendingMainPage = null;
	}

	// make sure it is added to the windows list
	if (!_windows.Contains(window))
		AddWindow(window);

	return window;
}

Even more, is revealed if we take a look at IApplication. It looks like we will be able to OpenWindow, CloseWindow, and CreateWindow.

/// <summary>
/// Class that represents a cross-platform .NET MAUI application.
/// </summary>
public interface IApplication : IElement
{
	/// <summary>
	/// Gets the instantiated windows in an application.
	/// </summary>
	IReadOnlyList<IWindow> Windows { get; }

	/// <summary>
	/// Instantiate a new window.
	/// </summary>
	/// <param name="activationState">Argument containing specific information on each platform.</param>
	/// <returns>The created window.</returns>
	IWindow CreateWindow(IActivationState? activationState);

	void OpenWindow(IWindow window);

	/// <summary>
	/// Requests that the application closes the window.
	/// </summary>
	/// <param name="window">The window to close.</param>
	void CloseWindow(IWindow window);

	/// <summary>
	/// Notify a theme change.
	/// </summary>
	void ThemeChanged();
}

Here we can see one of the commits which is an initial implementation of Windows support: https://github.com/dotnet/maui/commit/6743036c67c4c19263ca180deb7523e0750b4820

From within .NET MAUI codebase, we can see a sample of how to use multiple windows in the .NET MAUI samples projects, look for MultiWindowPage.

public partial class MultiWindowPage : BasePage
{
	static int windowCounter = 1;

	public MultiWindowPage()
	{
		InitializeComponent();

		label.Text = "Window Count: " + (windowCounter++).ToString();
	}

	void OnNewWindowClicked(object sender, EventArgs e)
	{
		Application.Current.OpenWindow(new Window(new MultiWindowPage()));
	}

	void OnCloseWindowClicked(object sender, EventArgs e)
	{
		var window = this.GetParentWindow();
		if (window is not null)
			Application.Current.CloseWindow(window);
	}
}

It’s awesome to see that we have multiple windows going on, this is good for multiple reasons including support for Desktop Applications and IPad. This Window functionality will probably become more useful in future iOS/Android OS releases if the native OS build out the Windowing functionality further.

The current preview of .NET MAUI does not have the code changes for IWindow support so you will either need to download the .NET MAUI source or wait until we get our next preview release.

Here are the windows events all explained: https://github.com/dotnet/maui/issues/1720

Summary

  • Windows is a new concept built in .NET MAUI
  • Windows will allow you to open and close Windows
  • It looks like it will be very useful for Desktop applications

This is only a brief look at IWindow in .NET MAUI, there will be a lot more info to come as I discover more and the .NET team builds out the functionality further. I’m looking forward to sharing the knowledge. If you have any questions or need some .NET MAUI Consulting and XAM is here to help.