Handle
A 'duck' is simply a handle that can be used to control some parameter of a layer.
For example, a circle will have one duck to control the position of its centre, and another to control its radius. Drag the ducks around to change the parameters.
More precise control of the ducks can be achieved using the cursor keys. The cursor keys will move the currently selected duck one pixel at a time, or ten pixels at a time if the shift key is pressed.
Why 'ducks'? According to Philip J. Schneider:
Back in the days before computers, architects, engineers, and artists would draw their designs for buildings, roads, machine parts, and the like by using pencil, paper, and various drafting tools. These tools included rulers and T-squares for drawing straight lines, compasses for drawing circles and circular arcs, and triangles and protractors for making precise angles.
Of course, a lot of interesting-shaped objects couldn't be drawn with just these simple tools, because they had curved parts that weren't just circles or ellipses. Often, a curve was needed that went smoothly through a number of predetermined points. This problem was particularly acute in shipbuilding: although a skilled artist or draftsman could reliably hand-draw such curves on a drafting table, shipbuilders often needed to make life-size (or nearly life-size) drawings, where the sheer size of the required curves made hand drawing impossible. Because of their great size, such drawings were often done in the loft area of a large building, by a specialist known as a loftsman. To aid in the task, the loftsman would employ long, thin, flexible strips of wood, plastic, or metal, called splines. The splines were held in place with lead weights, called ducks because of their resemblance to the feathered creature of the same name.
Also, according to wikipedia:
Before computers were used for creating engineering designs, drafting tools were employed by designers drawing by hand. To draw curves, especially for shipbuilding, draftsmen often used long, thin, flexible strips of wood, plastic, or metal called a spline (or a lath, not to be confused with lathe). The splines were held in place with lead weights (called ducks because of their duck like shape).
