To select shapes in a Mac SwiftUI canvas, read the pointer in the canvas view’s coordinate space, then reverse the transform used for drawing. With zoom and pan, subtract the pan offset and divide by zoom before testing your shapes.

This tutorial builds a small desktop canvas containing two synthetic rectangles. You will highlight the rectangle under the pointer and select a rectangle when a pointer gesture ends, including after changing zoom or pan. All drawing data stays in memory.

Which coordinate space should receive pointer events?

Keep three spaces distinct: the coordinate system storing your shapes, the local coordinate system of the canvas view, and any window coordinates supplied by AppKit. A point is useful only when you know which space it belongs to.

SwiftUI’s onContinuousHover(coordinateSpace:perform:) supplies pointer locations in the requested coordinate space; its default is .local. It delivers .active(location) while the pointer is inside the view and .ended when it leaves. Attach it directly to the canvas so that its local space matches the drawing viewport. Apple’s continuous hover documentation.

A named coordinate space is useful when an ancestor intentionally owns the interaction region. Both hover and gesture handlers should then request that same named space. Apple defines .local, .global, and .named(_:) as layout coordinate spaces; naming a space does not define the application’s drawing model. Apple’s coordinate-space documentation.

That distinction matters because Canvas performs immediate mode drawing. Apple explicitly documents that individual drawn elements, including symbols, do not receive their own interactivity. The supported approach here is to handle interaction on the containing canvas and perform model hit testing yourself. Apple’s Canvas documentation.

How do you reverse the drawing transform?

Choose an explicit drawing convention. In this example, model coordinates increase rightward and downward. Zoom scales model units, and pan is an offset measured in canvas-local points. The forward mapping and its inverse are:

viewX = modelX * zoom + panX
viewY = modelY * zoom + panY

modelX = (viewX - panX) / zoom
modelY = (viewY - panY) / zoom

These equations follow from that chosen mapping. To draw it, translate the graphics context by pan and then scale by zoom. Apple documents that these methods update the context’s transform and affect subsequent drawing operations. Reversing their order changes the mapping, so keep the inverse paired with the exact drawing sequence. Translation documentation, scaling documentation.

For an illustrative check, use zoom 2, pan (40, 30), and a local pointer position (180, 110). Subtracting pan gives (140, 80); dividing by zoom gives model position (70, 40). Drawing that model point returns it to (180, 110). A rectangle whose model frame contains (70, 40) should therefore be hit.

Keep this calculation in one shared type. The renderer and interaction code should consume the same mapping rather than maintaining separate offset formulas. The example accepts only finite, positive zoom values; zero cannot be inverted.

Build a complete hover-and-selection example

Use a macOS SwiftUI app project with a deployment target of macOS 14 or later. That target supports the CoordinateSpaceProtocol overload used by the hover modifier. Add the following model file, then replace the project’s generated app declaration with the second file. Keep exactly one @main declaration in the target. Hover API availability and signature.

Define model geometry and its mapping

Create CanvasGeometry.swift with the following complete contents. Rectangle frames remain in model coordinates throughout the interaction.

import SwiftUI

struct CanvasMapping {
    let zoom: CGFloat
    let pan: CGSize

    init(zoom: CGFloat, pan: CGSize) {
        precondition(zoom.isFinite && zoom > 0)
        precondition(pan.width.isFinite && pan.height.isFinite)
        self.zoom = zoom
        self.pan = pan
    }

    func modelPoint(from viewPoint: CGPoint) -> CGPoint {
        CGPoint(
            x: (viewPoint.x - pan.width) / zoom,
            y: (viewPoint.y - pan.height) / zoom
        )
    }

    func apply(to context: inout GraphicsContext) {
        context.translateBy(x: pan.width, y: pan.height)
        context.scaleBy(x: zoom, y: zoom)
    }
}

struct CanvasItem {
    let id: Int
    let frame: CGRect
}

The geometry and mapping are Swift structures. Their value semantics make it straightforward to pass the current values into drawing and hit-testing code without introducing another shared mutable object. Swift’s language guide documents structures as value types. Structures and Classes.

Draw and interact with the same viewport

Create PointerCanvasApp.swift, replacing the generated app declaration. This complete file uses the types from CanvasGeometry.swift.

import SwiftUI

@main
struct PointerCanvasApp: App {
    var body: some Scene {
        WindowGroup {
            PointerCanvasView()
        }
    }
}

struct PointerCanvasView: View {
    private let items = [
        CanvasItem(id: 1, frame: CGRect(
            x: 40, y: 30, width: 120, height: 80)),
        CanvasItem(id: 2, frame: CGRect(
            x: 120, y: 70, width: 120, height: 80))
    ]

    @State private var zoom: CGFloat = 2
    @State private var pan = CGSize(width: 40, height: 30)
    @State private var pointerInView: CGPoint?
    @State private var selectedID: Int?

    private var mapping: CanvasMapping {
        CanvasMapping(zoom: zoom, pan: pan)
    }

    private func hit(at viewPoint: CGPoint) -> Int? {
        let point = mapping.modelPoint(from: viewPoint)
        return items.reversed().first {
            $0.frame.contains(point)
        }?.id
    }

    var body: some View {
        let currentMapping = mapping
        let hoveredID = pointerInView.flatMap { hit(at: $0) }

        VStack(alignment: .leading, spacing: 12) {
            HStack {
                Button("Zoom out") {
                    zoom = max(0.5, zoom / 2)
                }
                Button("Zoom in") {
                    zoom = min(4, zoom * 2)
                }
                Button("Pan right") {
                    pan.width += 20
                }
                Button("Reset view") {
                    zoom = 2
                    pan = CGSize(width: 40, height: 30)
                }
                Text("Selected: \(selectedID.map { String($0) } ?? "none")")
            }

            Canvas { context, _ in
                var drawing = context
                currentMapping.apply(to: &drawing)

                for item in items {
                    let path = Path(item.frame)
                    drawing.fill(
                        path,
                        with: .color(
                            item.id == hoveredID ? .orange : .blue
                        )
                    )
                    if item.id == selectedID {
                        drawing.stroke(
                            path,
                            with: .color(.black),
                            lineWidth: 2 / currentMapping.zoom
                        )
                    }
                }
            }
            .background(Color.white)
            .contentShape(Rectangle())
            .onContinuousHover(coordinateSpace: .local) { phase in
                switch phase {
                case .active(let point):
                    pointerInView = point
                case .ended:
                    pointerInView = nil
                }
            }
            .gesture(
                DragGesture(minimumDistance: 0, coordinateSpace: .local)
                    .onEnded { value in
                        selectedID = hit(at: value.location)
                    }
            )
            .clipped()
        }
        .padding(16)
        .frame(minWidth: 640, minHeight: 420)
    }
}

The canvas uses a rectangular interaction region, including empty space between shapes. contentShape(Rectangle()) defines the view’s hit-testing shape; the separate frame.contains test decides which synthetic item is under the pointer. Apple’s content-shape documentation.

Selection uses a zero-minimum-distance DragGesture and reads the ending event’s location. Its defined behavior is to select the rectangle under the release position, even if the pointer moved while pressed. This is a release-position selector, rather than a strict click recognizer. Apple documents the gesture’s distance threshold and coordinate-space configuration, and identifies location as the current event position. DragGesture, DragGesture.Value.

The items are drawn in array order and tested in reverse order. Thus rectangle 2 wins where the two overlap, matching this example’s visual stacking. Clicking empty space deliberately clears selection. Shape frames and array contents never change.

The view owns pointer and selection state, while the renderer reads it. Apple recommends declaring private state in the highest view that needs it and passing bindings when descendants require write access. If a parent later owns the item collection or selection, move ownership there and pass the required values or bindings into this view. Apple’s state documentation.

How can you check the mapping before adding more gestures?

Start with known model points and calculate their expected local positions. These checkpoints use the sample’s initial zoom and pan:

Model point Expected canvas-local point Expected result
(70, 40) (180, 110) Rectangle 1
(140, 90) (320, 210) Rectangle 2, in the overlap
(10, 10) (60, 50) No rectangle

The local positions are measured from the canvas itself; the toolbar and outer padding are outside that space. Move across both rectangles, release over their overlap, and then try Zoom out and Pan right. The highlight should follow the visible geometry at each setting.

The illustrative inverse calculation and 12 forward/inverse round-trip checks were executed using Python arithmetic. A Swift compiler and macOS UI runtime were unavailable in the research environment, so the Swift application was inspected but not compiled or interactively tested.

If a check fails, use the symptom to narrow the cause:

  • A constant offset suggests that the input handler belongs to a different view or that pan was omitted.
  • An error that grows with distance suggests a missing scale conversion or a mismatched transform order.
  • A vertically mirrored result suggests an extra y-axis flip or an unconverted AppKit point.
  • Incorrect selection only in overlapping regions suggests inconsistent drawing and hit-test order.

Store the raw local hover point, as the example does, and derive the model point from the current mapping. That avoids retaining a model-space hover position calculated under an earlier zoom. Selection uses the gesture event’s own location, so it does not depend on a previous hover callback.

What changes when the event comes from AppKit?

If an existing custom AppKit view already owns the mouse interaction, convert the event at that boundary before applying the drawing inverse. NSEvent.locationInWindow normally uses the associated window’s base coordinates. Apple notes that an event with a nil window can instead contain screen coordinates, so establish that the event belongs to the receiving view’s window first. Apple’s event-location documentation.

Inside that receiving NSView, the expression convert(event.locationInWindow, from: nil) converts a window point into the view’s local coordinates. Apple documents nil as the window-coordinate source for this conversion. Only then apply the inverse of the transform used by that view’s renderer. Apple’s point-conversion documentation.

Check orientation at the view boundary. An ordinary AppKit view has a lower-left origin with y increasing upward; a view whose isFlipped returns true has an upper-left origin with y increasing downward. A custom AppKit drawing surface can adopt the latter convention to match this model. Let view conversion handle differences between view coordinate systems, and avoid adding another flip afterward. Apple’s flipped-coordinate documentation.

When exposing such a view through NSViewRepresentable, keep AppKit event handling in the represented view and communicate results through an explicit callback or coordinator. Refresh that connection during updates and clean up owned resources during teardown. SwiftUI controls the represented view’s layout, so do not set its frame or bounds to compensate for mapping errors. Apple’s representable lifecycle documentation.

For the pure SwiftUI example, keep interaction attached to the canvas. If an AppKit surface replaces it, give that surface ownership of the interaction region. Coordinate conversion should translate an event supplied by its existing owner; it should not require forwarding a second copy into another competing selection handler.

How should the mapping evolve for a larger canvas?

The sample deliberately supports uniform positive zoom and translation. If you add rotation, nonuniform scaling, or a different drawing origin, replace the simple inverse with the inverse of the full drawing transform. Reject noninvertible mappings before interaction, and retain forward/inverse checks with known points.

For future item dragging, record the item and model-space grab offset at gesture start. Update that same item throughout the gesture; choose explicitly whether the viewport mapping stays fixed or changes during the drag. Repeating hit testing to choose a new item on every movement would change the interaction’s ownership whenever shapes overlap.

Use one viewport coordinate space, one drawing mapping, and one matching inverse. Once those agree, hover and selection can operate on stable model geometry while zoom and pan change how that geometry appears.

Sources