Irregular Shaped Rooms

Rectangular or any type of regularly shaped room in roguelike games has always bothered me, so today, instead of working I came up with a simple way to generate irregular polygons, or more "natural" looking caves. A few examples are shown below.

How it works

The basic concept is simple enough:

1. Enclose an area in 4 rectangles (left picture),
2. Randomly choose at least 1 point in each of the rectangles (second from left),
   1. Order the points in Rectangle A by their X value, from low to high, add them to a list.
   2. Order the points in Rectangle B by their Y value, from low to high, add them to a list.
   3. Order the points in Rectangle C by their X value, for high to low, add them to a list.
   4. Order the points in Rectangle D by their Y value, from high to low, add them to a list.
3. Starting with the first point in the main list use Bresenham's Line algorithm to connect it to the next point, and so on, until you reach the end of the list. Connect the last point to the first point of the list, to completely enclosed the shape (third from left picture). The way the points have been arranged means they are connected in a clockwise direction (right picture).

To fill in the room, sort all grids that were 'drawn' by the line algorithm from lowest to highest x value for each y value. Then scan from left to right, flipping from outside to inside (or vice versa) whenever you hit a new edge. You'll need to record which edge corresponds to which set of points as you generate the lines above, to avoid two consecutive grids which belong to the same edge incorrectly flipping the outside/inside bit.

Alternately you can use floating point math to determine the polygon interior. The C code below (from http://www.ecse.rpi.edu/Homepages/wrf/Research/Short_Notes/pnpoly.html#The%20C%20Code) avoids the need for Bresenham's Line algorithm:

int pnpoly(int nvert, float *vertx, float *verty, float testx, float testy)
{
 int i, j, c = 0;
 for (i = 0, j = nvert-1; i < nvert; j = i++) {
   if ( ((verty[i]>testy) != (verty[j]>testy)) &&
      (testx < (vertx[j]-vertx[i]) * (testy-verty[i]) / (verty[j]-verty[i]) + vertx[i]) )
      c = !c;
 }
 return c;
}

Notes

Step 2, really is the most important part, so here it is again.

Moving from left to right, the points in Rectangle A are joined, then moving from top to bottom the points in Rectangle B are joined, then moving from right to left Rectangle C's points are joined and finally, from bottom to top Rectangle D's points are joined. Basically, we move around the rectangles clockwise joining the points as we go. This is the most important bit to get right, else the polygon produced is not enclosed.

Messing around with the settings can produce some surprising and varied shapes. Have fun!