Started work on a procedural Apparance terrain system in UE5, mostly using Images/Noise for heightmaps, and Frame+ for foliage population.
Here’s the current result, about 5 days of work. I’ll break this up into a writeup for what I’ve done so far, and then hopefully update the thread with continuing work on it.
This started as a challenge from a friend to recreate this kind of rolling landscape seen in Wartales, supporting non-gridded buildings on arbitrary terrain:
Noise-based terrain is nothing new, and I knew it wouldn’t be hard to make it look good with Unreal 5 & Quixel, but non-flat, non-grid-based building positioning is a fun challenge.
Firstly, I passed through some noise to the Heightmap Operator, then placing cubes/bushes with a Grid of Frames which followed a duplicated noise pattern with fewer iterations (smoother)
Then passing through the Frames as the Noise’s Sections, I could add multiple tiles of this. (I didn’t flip an axis on the frame, so the noise isn’t quite tiling here). I passed through a new Noise pattern with a bit of “edge detection”. This is really like making a single spike in a curves adjustment in photoshop, selecting a single band within the noise. This is passed through a Mask+ Op to cull all but the thin strip’s frames.
This was used to drive placement of bushes (for now). Resizing, Offseting, and Turning these frames after Masking made them naturally placed enough (in the video they aren’t properly set to the heightmap, as it was still using a “smoother” noise than the terrain).
The noise information was Colourised, and passed through as vertex colours to the Unreal shader. This would then be usable for assigning different materials to a single shader
This vertex colour map was easy to then assign to UE5 materials. I set up some fast and messy voronoi-based breaking up of worldspace coordinates for patches of angled UVs. Otherwise the shader is a bog standard blend of Quixel materials I could quickly iterate through. The fields look a bit like bacon…
The second part shows the noise maps used for the foliage are distinct from the terrain noise maps, though they could influence each other later if needbe.
I threw in some standard trees from a previous project, remembered to enable Nanite, and made the foliage-placing function more general. As well as thickness of the strip, it needed to support asset names, debug cube preview option (always), and some randomising amounts for scaling. I added a third Noise to optionally cull a random pattern of them as well, as we don’t want a solid wall of trees
It’s a shame not to utilise Apparance’s runtime capabilities, so at this point I deferred the whole thing.
I made an ApparanceEntity called “World”, which is simply an Apparance object nested within an Unreal BP. This lets me rebuild the Apparance procedures during play with whatever parameters passed through. These can be nested, so there can be a whole series of async nested procedures handled 
, eg the trees could be deferred to load cutouts if too far from the camera, reloading separately to the terrain.
I create some spheres in the scene with Actor Tags called controller, I could later defer the trees, maybe to be replaced with cutouts when the viewer is very far, etc.
The BP is linked to the correct Apparance proc, and the Construction Script looks for spheres placed in the scene with Actor Tags. These will be my “controllers” to affect the scene and pass player input/world info through to the procedures.
The “Handle Controller Move” is run on Construction’s end, but also per Event Tick. This checks if the controller is moved, scaled, or rotated at runtime, and if so rebuilds the Apparance proc with the controller’s new transform.
(I’ve put a “Demo Mode” bool here, which makes the updates happen much more frequently, just to record smoother videos.)
A demo of this: the game is running, and I’ve selected the ball to manipulate via the standard UE gizmos. This updates the terrain’s frame with the ball’s position.
Side note: there are interesting inconsistent random effects happenings. Ones that I fixed later like the terrain textures, but ones that I think are not fixable like the Variations within the Resize+/Offset+/Turn+ foliage nodes. Might be nice if they could derive their random seeds consistently across all the frames 
. But for now I’m OK with the tree rotations being invariant.
The terrain is changing while the frame moves up and down, because Noise is using all 3 axes. I only want XY positioning to be used as I’m thinking about my “sectors” as XY coordinates for now, and this should simplify things later. After this video I cancelled out the frame’s Z coordinate before feeding it into the Noise section.
I changed the controllers to affect something within the sector instead of moving the whole thing, in this case an arbitrary shape (XY Vertices in a List).
This shows how geometry can be heightmapped to the terrain, and the terrain being impacted by the geometry. This is done by passing the shape through the Draw Polygon on the heightmap.
Having control both ways seems like a good option for the foundation of a building. I’ve blurred the image map after masking in the polygon.
At this point I realised that I really needed to see all the Image maps being processed, so I made a procedure to show that in the world.
This enabled me to be able to do more delicate Image processing, like the following “cliff erosion”. The “edge selection” in this picture was made from a blurred Polygon stamp, then used as a mask for noise that modulated a the blurred height stamp.
I also added a cliff texture to any non-flat normals on the terrain shader.
That culminated the first weekend’s go at this. I’ll post the rest soon!
Water needed to be added, which was a case of another Noise and Composite op.
A windmill assset from a previously-bought asset pack was added as the building placeholder, and the “cliff erosion” procedure was applied to the cliffs made with the “water mask”. All the piecemeal generative systems are starting to already make nice emergent scenes like this (very inefficient but pretty) windmill bay thing:
The terrain was looking a little odd, so at this point it got overhauled with slightly larger mapping and better textures. Some simple world-position lerping was added to mask in some wet and dry sand at the appropriate level. The water is just a plane added at a set “water level” height.
The height map preview was very useful. Since it’s just a general procedure to display any floating Image map, I added ones for the other Image maps being used.
Note the 3-colour gradient-Colourised land zones from the 3rd post, with the “green” (1.0) being erased down to “red” (0.0), made by subtracting the “building plot” height map.
In the video you can see that because of this, at low heights, the subtraction isn’t strong enough and there are still “blue” (0.5) areas. This means that there’s a chance for foliage to spawn if the building isn’t high enough. This is easily fixed by scaling way up the Composite subtraction amount:
I decided to waste some time with the water, and made a new Image map for a “coastal refraction”. The water map is Blurred to create a depth contour. Sadly there is an issue here with my approach: Major blurs do not cross the boundaries of a map segment; that is, segments are blurred independently and so the top-left segment here doesn’t know that there should be a blurred cliff in its neighbour.
I can think of some overdesigned remedies, but for this purpose I turned down the blur amounts significantly and it’s enough for the wave effect. Flow Maps are often used for things like moving rivers, and are a nice shader trick, but always very fiddly to get just right.
Here I’ve scaled way up the wave speed, made the normals more obvious, and disabled a layer of randomised waves to demonstrate them:
Next up, pathing between “Controllers”. I’d like there to be an aesthetically snaking path, and there are some nice ways of doing this - particularly I’d like to eventually implement the “Least Elevation Distance” function (§4.2.3) of http://www.citygen.net/files/Citygen-Thesis.pdf (Sort of a ease-in ease-out/s-curve in height, which makes sense for flatter junctions:
To make things easier for now however, and because paths are often formed by non-elevation influences, I figured to just make a straight connecting line, split up recursively into bezier segments, which bulge out sideways in alternating randomised amounts to create a hopefully natural-ish looking meandering path.
Here’s the first working pass of this attempt, with lots of debug visualisations turned on (absolutely necessary for me!)
The length between Controllers (Flattened, as I don’t care about Z-differences and will push the road “up” onto the terrain with Heightmap later) is recursively split up into segments of a specified length.
The blue vector lines going up are calculated for each segment end by adding the segment’s length to its Z. The segment and the “segment up” are then enough to use for a cross product that returns a side vector. The segment count is then counted, and every odd iteration the side vector is inverted.
The side vectors are moved to the middle of each segment, and rendered here with the bright green debug lines.
A bezier is then added with ends at segment ends, and both handles at the green side vector’s ends.
Next I randomised the segment length. As it’s recursively filling, this can’t overflow and works nicely. I also randomised the side vector scale. It’s a little “regular” but starting to look good already.
This seems a bit complex, but it was made at first without the earlier “Randomise…” section, imagine that replaced with a single float value. If there is more total distance than the desired segment length, we append a new Curve to a list. If it’s reached the end, we return the list:
The Random.FromWorldXYZ is a cheap utility script that takes a vec3 position, Quad, or Frame, and simply returns the xyz product (avoiding any 0’s).
The Curves themselves are generated like so (I’ve squashed it all a bit vertcally for a screenshot, sorry!):
Of course there are endless little things to add on top. Having the handles be randomly different would help a lot, but it works for now.
The curve is then pushed either “side” by the side vector to create two sides of a path (width randomised, too):
This is quicker than doing real bezier offsetting, and relies on the fact that the “windy” paths will always be basically a straight line. Long sideways bends may become slightly skewed, but it works well enough for now.
A Polygon Image “path” is created from these two path sides, and subtracted just as the building base was, to ensure a texture with no foliage (see the Heightmap preview squares in the vid above).
This is pretty good, but using the built-in Polygon Extrude op isn’t enough to turn the two path sides into a poly. It’s filling the shape with triangless along one direction, while we need something more akin to “bridge edges” from a 3d package, constructing tris neatly along two paths of equal segment count.
Something like…this!:
Thankfully I’d already written this bridging procedure before. Both curves are iterated through:
First, a “Quad” list of verts are created. Then, this is passed through to a Quad making operator, which creates the two appropriate tris (Colour Triangle Op), with a UV Frame assigning approximate UVs.
The bridge op also has some helper functionality for end caps and such, but not needed here.
This bridged poly is great, and the Curves’ smoothness control affect its resolutions, but I need subdivisions width-wise, in order to camber up the mesh, or add natural-feeling noisy Z-displacement to the mesh.
This is achieved by rerunning the bridge operation, with replicated Curves pushed laterally along that sideways Vec3 again. My inputs for this are subdivision resolution and path width.
The Maths.Vec3Arithmetic.Vec3In node used above to translate shape lists (and many things elsewhere) is a utility node made up of 3 of these nodes.
They act as any of the regular Maths Operators, but on the XYZs of a whole List of Vec3s instead of single floats. eg - take in a shape of a circle (list of Vec3s) and add a float of 4.0 to the Z to raise the shape in worldspace by 4 units.
As they only take a single float operand, this utility runs them 3 times for each XYZ component of a Vec3. eg take in a circle shape and add a Vec3 of (4,3,0) to translate the shape by that vector along the XY in worldspace.
The underlying proc is a mess as it’s simply a big switch case of the regular float maths operators iterating on a list… hopefully we’ll have Maths ops for Vec3s/lists some day 
Anyway. this allows for a lot of nice operations on shapes like our path. The same side translation is done with the sideways vector, to create nice offset curves for fences:
The fence pieces are from Quixel, and placed with an old proc I’ve made for iterating the Path to Frame op along a path, also Heightmapped, although their tilts will need adjusting:
The path polygon Image is then blurred, and applying this as a Heightmap to it gives a cambered effect (extremely exaggerated):
The UV mapped polys no longer work with the path’s width-ways subdivision, as the UVs are split up into too-small quads. I’ve replaced the UV mapping with a top-down worldspace projection in the shader, with some classic multiplied noise to break it up and remove repetition. I added this to the terrain texture projections as well (left side showing the noise map):
I pass through the same blurred shape’s “height map” mask to the vertex colours of the mesh. This is used in the shader as a mask for opacity, dithered with the Dither Temporal AA node for cheap blending with the terrain – no more sharp edge mesh/land intersection.
That about covers the paths, much more could be done to break things up in the future. I check one of the “widths” between fences, and if it’s narrow enough, only one random side will get a fence.
Finally for now, I made a procedure that places many sectors for background tiles. These are built up around the “inner square” as 4 corner squares growing outwards, each time with an increased “distance” value.
Distance drives obvious things like Image and Noise resolutions, scales down the density of certain foliage, etc.
Nanite can collapse the few-triangled leaves, so I swapped them out with a simple stretched sphere with the same leaves texture at higher distances, which worked great (Nanite tri collapse settings set here to exaggerated values):
That’s it! Thank you for having a look… this took about half the time to write up as it did just to make the damn thing so far 
The biggest limitation currently is that the controllers are locked to a single sector at a time. They can change sector, but the pathing inbetween is only currently looking at only one Image map.
Another issue is that the image maps are all processed at once with any change. When moving the controllers around to say, stamp a road, it’s going through all the noise, blurs, compositing etc for the terrain underneath. I’d love to split this out and cache things, maybe even cache terrain per world coordinates.
Next steps that come to mind are
Currently you can hit “Play” and fall into the world and run around as the terrain generates collision, but there’s nothing to do. Maybe I’ll think about coming at this from a game angle next to drive features.
