To keep a precise vector boundary while blur fades across an abstract design, apply the blur as a Smart Filter and put a gradient on its filter mask. Keep the vector mask separate. This Mac desktop workflow preserves independent control of the outline, artwork, and blur.

The example below creates overlapping rectangles and an ellipse, then contains them inside a crisp oval. Detail stays sharp on the left and becomes blurred toward the right. All dimensions and filter values are illustrative starting points.

Which mask controls the result you want?

Photoshop’s masks have different jobs. A vector mask defines visibility with a path. A layer mask controls the artwork’s visibility with grayscale values. A Smart Filter mask controls where filter effects appear. Adobe documents these separately in its guides to vector masks, layer-mask editing, and Smart Filter masks.

Desired appearance Control to use What changes
A precise oval boundary Vector mask The visible outline
Artwork gradually disappearing Gradient on a layer mask Artwork visibility
Detail gradually becoming blurred Gradient on a Smart Filter mask Filter visibility
A softened outer boundary Vector-mask feathering Edge softness

For the main example, use the first and third controls together. A gradient on an ordinary layer mask creates a fade into the background; Adobe’s gradual blending tutorial demonstrates that result. It does not create blurred detail.

Build original abstract artwork with editable source shapes

Create a new document using File > New. For practice, choose a canvas of 1600 × 1000 pixels, RGB Color, 8 bits per channel, and a white background. Open Window > Layers and Window > Properties so you can inspect what is selected throughout the exercise.

Draw a large blue rectangle, a narrower orange rectangle crossing it, and a small yellow ellipse overlapping both. Use the Rectangle Tool and Ellipse Tool in Shape mode. Choose each shape’s Fill color in the options bar and omit the stroke. Keep each shape on its own layer; select the background before drawing the next shape if necessary to avoid extending an existing shape. Adobe explains the available drawing modes and fill controls in Shape, path, and pixel mode options.

Make the composition larger than the oval you intend to reveal. Include color boundaries on both sides of its center: these give you detail against which to judge the blur. A single uniform color would make an interior blur difficult to evaluate.

In the Layers panel, select the three artwork layers, leaving the white background unselected. Choose Layer > Smart Objects > Convert to Smart Object, then name the resulting layer Abstract artwork. Adobe documents that this command can bundle multiple selected layers into one embedded Smart Object.

This is the first important editing boundary: the original shapes sit inside the Smart Object, while the following filter and masks belong to its outer layer. To revise the shapes later, choose Layer > Smart Objects > Edit Contents, edit the opened source document, and save it to update the composition. See Adobe’s Smart Object content-editing instructions.

Add blur and confine it with a vector boundary

Select the Smart Object’s artwork thumbnail. Choose Select > Deselect before applying the filter so an earlier selection does not constrain the operation. Choose Filter > Blur > Gaussian Blur, try a radius of 18 pixels, and confirm. This radius is an example, not a required setting. Adobe describes Gaussian Blur as an adjustable blur in its filter effects reference.

Check that the filter appears beneath the Smart Object in the Layers panel. Smart Filters preserve the source pixels and keep effects separately editable; Adobe explains this structure in Apply and manage Smart Filters.

Now add the outer boundary:

  1. Select the Abstract artwork layer.
  2. Choose the Ellipse Tool and switch its drawing mode to Path.
  3. Draw a wide oval through the overlapping shapes, leaving artwork beyond its perimeter.
  4. Choose Layer > Vector Mask > Current Path.
  5. Select the vector-mask thumbnail and keep its Density at 100% and Feather at 0 pixels for a crisp boundary.

Adobe documents path-based mask creation and these properties in Mask layers with vector masks.

The intended arrangement is now a Smart Object with a vector mask on its layer row, plus a filter-mask thumbnail beside Smart Filters below. The underlying shapes extend beyond the oval, giving the blur surrounding artwork to sample before the final boundary limits visibility. This arrangement is a practical application of the documented filter and mask controls.

Draw the gradient on the blur mask

Click the mask thumbnail beside Smart Filters, beneath the artwork layer. Do not select the vector-mask thumbnail or an ordinary layer-mask thumbnail. Adobe states that a filter mask is created automatically when a Smart Filter is applied, and that white reveals effects, black hides them, and gray blends them. See Mask Smart Filter effects.

Select the Gradient Tool. Set the foreground color to black and the background color to white, then choose the foreground-to-background gradient from the preset picker. Adobe’s layer-mask gradient tutorial documents this preset workflow; applying it to the selected filter mask follows Adobe’s documented use of painting and editing tools on filter masks.

Choose Linear as the gradient type and turn Reverse off. Drag horizontally from roughly one-third of the way across the oval to two-thirds of the way across it. The black starting side suppresses the blur, while the white ending side reveals it. Adobe’s gradient-fill documentation explains the gradient types and how drag direction and length control the transition.

Inspect the result before adding any further effects. You should see recognizable sharp color boundaries on the left, softened boundaries on the right, and the same crisp oval perimeter around both.

Treat the blur amount and the transition width as two separate decisions. If the right side is too soft, adjust the Gaussian Blur settings. If the change happens too abruptly, widen the gradient transition. For an illustrative comparison, a transition spanning about 500 pixels gives the blend more room than one spanning about 150 pixels; neither distance is universally preferable.

There is a useful limit to this method: a gradient filter mask blends the filtered result with the unfiltered result. It does not assign a different Gaussian Blur radius to every position. That distinction follows from Adobe’s description of filter masks as controls over effect visibility.

When you need blur strength to vary across the artwork

For a spatially graduated blur, use Field Blur instead of the masked Gaussian Blur approach. Remove the Gaussian Blur entry first to avoid combining two treatments unintentionally. With the Smart Object selected, choose Filter > Blur Gallery > Field Blur. Place a pin on the left with a blur value of 0 and another on the right with a stronger value, such as 18 pixels. Adjust their positions, review the preview, and confirm.

Adobe documents that multiple pins with different values create graduated blur, and that Blur Gallery effects can remain editable as Smart Filters. Keep the existing outer vector mask and leave the filter mask white when you want the pins alone to determine the blur distribution. See Use Blur Gallery depth-of-field effects.

What if you wanted a gradual transparency fade?

Keep the vector boundary, but target a regular layer mask for the gradient. Select the artwork layer, deselect any active selection, and click the layer-mask icon in the Layers panel. If an ordinary layer mask already exists, select it instead of adding another. Adobe documents the creation procedure in Add layer masks.

Draw a white-to-black gradient on that mask to retain the artwork at the white end and hide it at the black end. The vector mask continues to define the outline, while the grayscale mask controls the fade. Adobe’s mask-editing guide confirms that darker mask values increase transparency without changing the source artwork.

Use this variant when the desired endpoint is the background showing through. Use the Smart Filter mask when the endpoint should remain visible but blurred.

Diagnose the result before saving

Check the mask target first. A border around a thumbnail identifies the active editing target, as Adobe explains in Edit and refine existing layer masks.

Symptom Check Correction
The artwork disappears toward one side A regular layer mask may contain the gradient Undo that change and select the filter mask
The blur runs in the opposite direction Inspect the filter mask’s black and white ends Select that mask and use Invert in Properties
The oval perimeter looks soft Inspect the vector mask’s feather setting Return Feather to 0 pixels
Another filter also fades Inspect the Smart Filter stack The filter mask controls all filters on that Smart Object

Adobe documents filter-mask inversion and the shared-mask limitation in Mask Smart Filter effects. To inspect that mask directly on a Mac, Option-click its thumbnail; repeat to return to the artwork.

Preserve the remaining editing boundaries. Change the oval using the Direct Selection Tool; avoid Layer > Rasterize > Vector Mask, which removes its vector editability. Adobe explicitly documents this conversion boundary in Mask layers with vector masks. Keep the Smart Object intact as well: converting it to a regular layer removes the editable filter relationship, according to Apply and manage Smart Filters.

Save the master as a Photoshop document with layers retained, using File > Save As. Adobe’s saving guide explains the Layers option and using Save a Copy for a separate output.

Before finishing, verify three things in the saved master: the outline can still be reshaped, the blur can still be adjusted, and the original rectangles and ellipse remain accessible inside the Smart Object. That gives you a reusable composition whose boundary, detail, and transition can be revised independently.

Sources